May 20 21:40:43.909989 ip-10-0-141-67 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory May 20 21:40:43.910110 ip-10-0-141-67 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory May 20 21:40:43.910186 ip-10-0-141-67 systemd[1]: kubelet.service: Failed with result 'resources'. May 20 21:40:43.910474 ip-10-0-141-67 systemd[1]: Failed to start Kubernetes Kubelet. May 20 21:40:53.078830 ip-10-0-141-67 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. May 20 21:40:53.078843 ip-10-0-141-67 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1b3870b7e4164965a9baf03473052348 -- May 20 21:43:05.092849 ip-10-0-141-67 systemd[1]: Starting Kubernetes Kubelet... May 20 21:43:05.536896 ip-10-0-141-67 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 20 21:43:05.536896 ip-10-0-141-67 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 20 21:43:05.536896 ip-10-0-141-67 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 20 21:43:05.536896 ip-10-0-141-67 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 20 21:43:05.536896 ip-10-0-141-67 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 20 21:43:05.540443 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.540356 2565 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 20 21:43:05.543372 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543357 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 21:43:05.543372 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543372 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543377 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543380 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543383 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543386 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543389 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543393 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543395 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543398 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543401 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543403 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543406 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543408 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543411 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543414 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543416 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543419 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543421 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543423 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543426 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 21:43:05.543430 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543429 2565 feature_gate.go:328] unrecognized feature gate: Example2 May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543432 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543434 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543437 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543440 2565 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543443 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543446 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543449 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543451 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543454 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543456 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543459 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543461 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543464 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543466 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543468 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543471 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543473 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543476 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543478 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 21:43:05.543898 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543480 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543483 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543486 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543488 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543490 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543492 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543495 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543497 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543500 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543502 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543504 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543507 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543509 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543513 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543515 2565 feature_gate.go:328] unrecognized feature gate: Example May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543518 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543521 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543523 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543527 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 21:43:05.544360 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543531 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543534 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543537 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543539 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543542 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543544 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543546 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543555 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543558 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543561 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543563 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543566 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543580 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543583 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543586 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543588 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543593 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543597 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543600 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543603 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 21:43:05.544814 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543606 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543608 2565 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543611 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543614 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543616 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.543619 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544635 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544642 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544645 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544648 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544651 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544654 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544656 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544659 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544663 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544666 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544668 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544671 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544673 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 21:43:05.545326 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544677 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544681 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544684 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544687 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544689 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544692 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544695 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544697 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544700 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544702 2565 feature_gate.go:328] unrecognized feature gate: Example May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544705 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544707 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544710 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544712 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544715 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544717 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544719 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544723 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544725 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544728 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 21:43:05.545797 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544732 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544735 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544738 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544740 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544743 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544745 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544748 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544750 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544753 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544755 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544757 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544760 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544762 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544764 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544767 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544769 2565 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544772 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544774 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544777 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544779 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 21:43:05.546301 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544782 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544784 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544787 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544789 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544806 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544810 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544814 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544818 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544821 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544824 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544827 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544829 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544832 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544835 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544838 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544841 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544843 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544845 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544848 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 21:43:05.546785 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544850 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544853 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544855 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544858 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544861 2565 feature_gate.go:328] unrecognized feature gate: Example2 May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544863 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544866 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544869 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544871 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544873 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544876 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544878 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544881 2565 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.544883 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544945 2565 flags.go:64] FLAG: --address="0.0.0.0" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544958 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544969 2565 flags.go:64] FLAG: --anonymous-auth="true" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544974 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544978 2565 flags.go:64] FLAG: --authentication-token-webhook="false" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544981 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544986 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 20 21:43:05.547233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.544990 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545001 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545005 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545009 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545012 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545015 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545025 2565 flags.go:64] FLAG: --cgroup-root="" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545028 2565 flags.go:64] FLAG: --cgroups-per-qos="true" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545036 2565 flags.go:64] FLAG: --client-ca-file="" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545039 2565 flags.go:64] FLAG: --cloud-config="" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545042 2565 flags.go:64] FLAG: --cloud-provider="external" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545066 2565 flags.go:64] FLAG: --cluster-dns="[]" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545074 2565 flags.go:64] FLAG: --cluster-domain="" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545077 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545080 2565 flags.go:64] FLAG: --config-dir="" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545082 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545086 2565 flags.go:64] FLAG: --container-log-max-files="5" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545090 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545093 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545096 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545099 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545102 2565 flags.go:64] FLAG: --contention-profiling="false" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545105 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545108 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545111 2565 flags.go:64] FLAG: --cpu-manager-policy="none" May 20 21:43:05.547735 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545114 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545118 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545121 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545124 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545127 2565 flags.go:64] FLAG: --enable-load-reader="false" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545130 2565 flags.go:64] FLAG: --enable-server="true" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545137 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545143 2565 flags.go:64] FLAG: --event-burst="100" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545147 2565 flags.go:64] FLAG: --event-qps="50" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545150 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545153 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545156 2565 flags.go:64] FLAG: --eviction-hard="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545160 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545163 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545171 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545175 2565 flags.go:64] FLAG: --eviction-soft="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545178 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545180 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545185 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545187 2565 flags.go:64] FLAG: --experimental-mounter-path="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545190 2565 flags.go:64] FLAG: --fail-cgroupv1="false" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545193 2565 flags.go:64] FLAG: --fail-swap-on="true" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545196 2565 flags.go:64] FLAG: --feature-gates="" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545200 2565 flags.go:64] FLAG: --file-check-frequency="20s" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545203 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 20 21:43:05.548340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545207 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545210 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545213 2565 flags.go:64] FLAG: --healthz-port="10248" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545216 2565 flags.go:64] FLAG: --help="false" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545219 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-141-67.ec2.internal" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545222 2565 flags.go:64] FLAG: --housekeeping-interval="10s" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545225 2565 flags.go:64] FLAG: --http-check-frequency="20s" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545228 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545231 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545235 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545238 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545241 2565 flags.go:64] FLAG: --image-service-endpoint="" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545244 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545248 2565 flags.go:64] FLAG: --kube-api-burst="100" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545250 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545254 2565 flags.go:64] FLAG: --kube-api-qps="50" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545256 2565 flags.go:64] FLAG: --kube-reserved="" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545259 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545262 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545265 2565 flags.go:64] FLAG: --kubelet-cgroups="" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545268 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545271 2565 flags.go:64] FLAG: --lock-file="" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545279 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545282 2565 flags.go:64] FLAG: --log-flush-frequency="5s" May 20 21:43:05.548940 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545285 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545291 2565 flags.go:64] FLAG: --log-json-split-stream="false" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545294 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545297 2565 flags.go:64] FLAG: --log-text-split-stream="false" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545300 2565 flags.go:64] FLAG: --logging-format="text" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545303 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545306 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545309 2565 flags.go:64] FLAG: --manifest-url="" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545312 2565 flags.go:64] FLAG: --manifest-url-header="" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545316 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545320 2565 flags.go:64] FLAG: --max-open-files="1000000" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545324 2565 flags.go:64] FLAG: --max-pods="110" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545327 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545330 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545333 2565 flags.go:64] FLAG: --memory-manager-policy="None" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545336 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545339 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545342 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545345 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545353 2565 flags.go:64] FLAG: --node-status-max-images="50" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545356 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545360 2565 flags.go:64] FLAG: --oom-score-adj="-999" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545363 2565 flags.go:64] FLAG: --pod-cidr="" May 20 21:43:05.549539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545366 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545372 2565 flags.go:64] FLAG: --pod-manifest-path="" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545375 2565 flags.go:64] FLAG: --pod-max-pids="-1" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545378 2565 flags.go:64] FLAG: --pods-per-core="0" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545381 2565 flags.go:64] FLAG: --port="10250" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545384 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545386 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d2387a42f809e5e3" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545390 2565 flags.go:64] FLAG: --qos-reserved="" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545398 2565 flags.go:64] FLAG: --read-only-port="10255" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545402 2565 flags.go:64] FLAG: --register-node="true" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545405 2565 flags.go:64] FLAG: --register-schedulable="true" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545408 2565 flags.go:64] FLAG: --register-with-taints="" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545412 2565 flags.go:64] FLAG: --registry-burst="10" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545415 2565 flags.go:64] FLAG: --registry-qps="5" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545418 2565 flags.go:64] FLAG: --reserved-cpus="" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545421 2565 flags.go:64] FLAG: --reserved-memory="" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545424 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545427 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545430 2565 flags.go:64] FLAG: --rotate-certificates="false" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545433 2565 flags.go:64] FLAG: --rotate-server-certificates="false" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545436 2565 flags.go:64] FLAG: --runonce="false" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545439 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545442 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545445 2565 flags.go:64] FLAG: --seccomp-default="false" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545448 2565 flags.go:64] FLAG: --serialize-image-pulls="true" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545451 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 20 21:43:05.550159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545454 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545457 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545460 2565 flags.go:64] FLAG: --storage-driver-password="root" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545463 2565 flags.go:64] FLAG: --storage-driver-secure="false" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545466 2565 flags.go:64] FLAG: --storage-driver-table="stats" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545470 2565 flags.go:64] FLAG: --storage-driver-user="root" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545473 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545476 2565 flags.go:64] FLAG: --sync-frequency="1m0s" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545479 2565 flags.go:64] FLAG: --system-cgroups="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545482 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545487 2565 flags.go:64] FLAG: --system-reserved-cgroup="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545490 2565 flags.go:64] FLAG: --tls-cert-file="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545492 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545499 2565 flags.go:64] FLAG: --tls-min-version="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545502 2565 flags.go:64] FLAG: --tls-private-key-file="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545512 2565 flags.go:64] FLAG: --topology-manager-policy="none" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545515 2565 flags.go:64] FLAG: --topology-manager-policy-options="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545518 2565 flags.go:64] FLAG: --topology-manager-scope="container" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545521 2565 flags.go:64] FLAG: --v="2" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545526 2565 flags.go:64] FLAG: --version="false" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545529 2565 flags.go:64] FLAG: --vmodule="" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545534 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.545537 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545661 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545665 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 21:43:05.550792 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545668 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545671 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545674 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545677 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545679 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545682 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545684 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545687 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545691 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545694 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545698 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545700 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545705 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545708 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545711 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545714 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545717 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545719 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545722 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 21:43:05.551403 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545725 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545727 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545730 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545733 2565 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545741 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545744 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545747 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545749 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545752 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545755 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545757 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545760 2565 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545762 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545765 2565 feature_gate.go:328] unrecognized feature gate: Example May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545767 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545770 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545772 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545775 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545777 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545780 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 21:43:05.551888 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545783 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545785 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545787 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545790 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545792 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545796 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545798 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545801 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545804 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545807 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545809 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545812 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545815 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545817 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545820 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545822 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545825 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545834 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545837 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545840 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 21:43:05.552492 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545842 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545845 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545848 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545850 2565 feature_gate.go:328] unrecognized feature gate: Example2 May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545852 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545855 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545858 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545860 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545863 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545867 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545872 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545875 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545877 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545880 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545882 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545885 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545887 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545891 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545894 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 21:43:05.553299 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545897 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 21:43:05.553787 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545899 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 21:43:05.553787 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545902 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 21:43:05.553787 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545905 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 21:43:05.553787 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545907 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 21:43:05.553787 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.545910 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 21:43:05.553787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.546681 2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 20 21:43:05.554161 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.554143 2565 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 20 21:43:05.554196 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.554162 2565 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 20 21:43:05.554238 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554212 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 21:43:05.554238 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554217 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 21:43:05.554238 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554232 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 21:43:05.554238 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554236 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 21:43:05.554238 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554238 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554241 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554244 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554247 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554249 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554252 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554255 2565 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554257 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554260 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554262 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554265 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554267 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554270 2565 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554272 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554275 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554278 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554280 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554283 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554285 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554289 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 21:43:05.554362 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554293 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554295 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554298 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554300 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554303 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554306 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554308 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554311 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554313 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554315 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554318 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554320 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554323 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554325 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554328 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554333 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554337 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554341 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554344 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 21:43:05.554853 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554348 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554351 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554354 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554357 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554359 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554362 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554365 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554368 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554370 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554373 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554376 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554378 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554381 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554383 2565 feature_gate.go:328] unrecognized feature gate: Example May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554386 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554388 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554390 2565 feature_gate.go:328] unrecognized feature gate: Example2 May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554393 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554395 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554398 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 21:43:05.555336 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554400 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554403 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554406 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554408 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554411 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554414 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554416 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554420 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554422 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554425 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554428 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554430 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554433 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554436 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554439 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554441 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554444 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554446 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554448 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 21:43:05.555824 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554451 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554453 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554456 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554458 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.554463 2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554585 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554592 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554595 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554597 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554600 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554603 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554606 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554609 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554611 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554614 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554616 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 20 21:43:05.556277 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554619 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554621 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554624 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554626 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554630 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554635 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554638 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554641 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554643 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554646 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554649 2565 feature_gate.go:328] unrecognized feature gate: DualReplica May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554653 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554656 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554658 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554661 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554663 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554666 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554668 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554671 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 20 21:43:05.556683 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554673 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554676 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554678 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554680 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554683 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554685 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554688 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554690 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554692 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554695 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554697 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554700 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554702 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554705 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554707 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554710 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554712 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554715 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554717 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 20 21:43:05.557197 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554720 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554723 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554726 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554728 2565 feature_gate.go:328] unrecognized feature gate: NewOLM May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554730 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554733 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554736 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554739 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554742 2565 feature_gate.go:328] unrecognized feature gate: Example2 May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554745 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554747 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554750 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554752 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554755 2565 feature_gate.go:328] unrecognized feature gate: Example May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554757 2565 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554759 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554762 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554764 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554767 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554769 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 20 21:43:05.557668 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554772 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554774 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554776 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554779 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554781 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554784 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554786 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554789 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554791 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554794 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554796 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554799 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554801 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554804 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554806 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554809 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 20 21:43:05.558148 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:05.554812 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 20 21:43:05.558541 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.554816 2565 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 20 21:43:05.558541 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.555551 2565 server.go:962] "Client rotation is on, will bootstrap in background" May 20 21:43:05.561778 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.561764 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 20 21:43:05.563178 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.563167 2565 server.go:1019] "Starting client certificate rotation" May 20 21:43:05.563280 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.563263 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 20 21:43:05.563309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.563304 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 20 21:43:05.586430 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.586412 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 20 21:43:05.588244 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.588225 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 20 21:43:05.606924 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.606904 2565 log.go:25] "Validated CRI v1 runtime API" May 20 21:43:05.612363 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.612345 2565 log.go:25] "Validated CRI v1 image API" May 20 21:43:05.613626 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.613610 2565 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 20 21:43:05.616279 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.616264 2565 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 20 21:43:05.617476 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.617457 2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8cc82456-cd97-4e0e-bec5-bca18b22d655:/dev/nvme0n1p4 b7530f70-7b8a-4d2c-8b6d-96174916be05:/dev/nvme0n1p3] May 20 21:43:05.617533 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.617476 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 20 21:43:05.621766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.621659 2565 manager.go:217] Machine: {Timestamp:2026-05-20 21:43:05.621140615 +0000 UTC m=+0.402573858 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3157041 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2359ef1832eecbd657e7419874ac92 SystemUUID:ec2359ef-1832-eecb-d657-e7419874ac92 BootID:1b3870b7-e416-4965-a9ba-f03473052348 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:aa:8f:1d:00:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:aa:8f:1d:00:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:84:6b:e4:79:b7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 20 21:43:05.622517 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.622507 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 20 21:43:05.622622 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.622609 2565 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 20 21:43:05.624391 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.624366 2565 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 20 21:43:05.624523 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.624394 2565 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-67.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 20 21:43:05.624582 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.624533 2565 topology_manager.go:138] "Creating topology manager with none policy" May 20 21:43:05.624582 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.624541 2565 container_manager_linux.go:306] "Creating device plugin manager" May 20 21:43:05.624582 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.624549 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 20 21:43:05.625538 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.625527 2565 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 20 21:43:05.628288 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.628278 2565 state_mem.go:36] "Initialized new in-memory state store" May 20 21:43:05.628398 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.628389 2565 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 20 21:43:05.630591 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.630582 2565 kubelet.go:491] "Attempting to sync node with API server" May 20 21:43:05.630631 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.630595 2565 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 20 21:43:05.630631 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.630607 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 20 21:43:05.630631 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.630615 2565 kubelet.go:397] "Adding apiserver pod source" May 20 21:43:05.630631 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.630625 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 20 21:43:05.631832 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.631817 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 20 21:43:05.631925 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.631836 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 20 21:43:05.634613 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.634599 2565 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 20 21:43:05.636711 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.636698 2565 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 20 21:43:05.637589 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637565 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637596 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637606 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637614 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637622 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637628 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637645 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637650 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637657 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 20 21:43:05.637665 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637662 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 20 21:43:05.638029 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.637677 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 20 21:43:05.638082 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.638043 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 20 21:43:05.638851 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.638834 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 20 21:43:05.638918 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.638855 2565 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 20 21:43:05.642067 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.642054 2565 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 20 21:43:05.642150 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.642087 2565 server.go:1295] "Started kubelet" May 20 21:43:05.642252 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.642193 2565 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 20 21:43:05.642297 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.642250 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 20 21:43:05.642297 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.642292 2565 server_v1.go:47] "podresources" method="list" useActivePods=true May 20 21:43:05.642877 ip-10-0-141-67 systemd[1]: Started Kubernetes Kubelet. May 20 21:43:05.643596 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.643562 2565 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 20 21:43:05.644222 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.644190 2565 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 20 21:43:05.644222 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.644189 2565 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-67.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 20 21:43:05.644222 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.644218 2565 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-67.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 20 21:43:05.645285 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.645270 2565 server.go:317] "Adding debug handlers to kubelet server" May 20 21:43:05.648935 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.648032 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-67.ec2.internal.18b16418d74afdd2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-67.ec2.internal,UID:ip-10-0-141-67.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-67.ec2.internal,},FirstTimestamp:2026-05-20 21:43:05.642065362 +0000 UTC m=+0.423498604,LastTimestamp:2026-05-20 21:43:05.642065362 +0000 UTC m=+0.423498604,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-67.ec2.internal,}" May 20 21:43:05.651050 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.651031 2565 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 20 21:43:05.651159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651145 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 20 21:43:05.651724 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651709 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 20 21:43:05.651820 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651804 2565 volume_manager.go:295] "The desired_state_of_world populator starts" May 20 21:43:05.651820 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651815 2565 volume_manager.go:297] "Starting Kubelet Volume Manager" May 20 21:43:05.651923 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651912 2565 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 20 21:43:05.651976 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651971 2565 reconstruct.go:97] "Volume reconstruction finished" May 20 21:43:05.652026 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.651979 2565 reconciler.go:26] "Reconciler: start to sync state" May 20 21:43:05.652393 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652380 2565 factory.go:55] Registering systemd factory May 20 21:43:05.652456 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652396 2565 factory.go:223] Registration of the systemd container factory successfully May 20 21:43:05.652583 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.652545 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:05.652667 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652636 2565 factory.go:153] Registering CRI-O factory May 20 21:43:05.652667 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652654 2565 factory.go:223] Registration of the crio container factory successfully May 20 21:43:05.652733 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652709 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 20 21:43:05.652733 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652728 2565 factory.go:103] Registering Raw factory May 20 21:43:05.652854 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.652742 2565 manager.go:1196] Started watching for new ooms in manager May 20 21:43:05.653287 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.653171 2565 manager.go:319] Starting recovery of all containers May 20 21:43:05.653866 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.653830 2565 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 20 21:43:05.653963 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.653938 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-67.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 20 21:43:05.661779 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.661645 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-85ndg" May 20 21:43:05.663508 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.663494 2565 manager.go:324] Recovery completed May 20 21:43:05.667491 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.667476 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-85ndg" May 20 21:43:05.667835 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.667823 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 21:43:05.672423 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.672409 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientMemory" May 20 21:43:05.672478 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.672437 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasNoDiskPressure" May 20 21:43:05.672478 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.672448 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientPID" May 20 21:43:05.673376 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.673362 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" May 20 21:43:05.673376 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.673376 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 20 21:43:05.673476 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.673392 2565 state_mem.go:36] "Initialized new in-memory state store" May 20 21:43:05.676054 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.676042 2565 policy_none.go:49] "None policy: Start" May 20 21:43:05.676106 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.676058 2565 memory_manager.go:186] "Starting memorymanager" policy="None" May 20 21:43:05.676106 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.676067 2565 state_mem.go:35] "Initializing new in-memory state store" May 20 21:43:05.727151 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727130 2565 manager.go:341] "Starting Device Plugin manager" May 20 21:43:05.727278 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.727164 2565 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 20 21:43:05.727278 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727174 2565 server.go:85] "Starting device plugin registration server" May 20 21:43:05.727415 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727404 2565 eviction_manager.go:189] "Eviction manager: starting control loop" May 20 21:43:05.727455 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727418 2565 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 20 21:43:05.727558 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727536 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 20 21:43:05.727672 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727662 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 20 21:43:05.727672 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.727672 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 20 21:43:05.728149 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.728129 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 20 21:43:05.728214 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.728171 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:05.739976 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.739955 2565 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 20 21:43:05.741117 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.741095 2565 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 20 21:43:05.741186 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.741129 2565 status_manager.go:230] "Starting to sync pod status with apiserver" May 20 21:43:05.741186 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.741147 2565 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 20 21:43:05.741186 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.741156 2565 kubelet.go:2451] "Starting kubelet main sync loop" May 20 21:43:05.741297 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.741197 2565 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 20 21:43:05.744831 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.744811 2565 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 20 21:43:05.828121 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.828040 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 21:43:05.829234 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.829219 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientMemory" May 20 21:43:05.829289 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.829247 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasNoDiskPressure" May 20 21:43:05.829289 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.829258 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientPID" May 20 21:43:05.829289 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.829281 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-67.ec2.internal" May 20 21:43:05.838761 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.838745 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-67.ec2.internal" May 20 21:43:05.838830 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.838765 2565 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-67.ec2.internal\": node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:05.841451 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.841434 2565 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal"] May 20 21:43:05.841519 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.841492 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 21:43:05.847116 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.847101 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientMemory" May 20 21:43:05.847183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.847132 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasNoDiskPressure" May 20 21:43:05.847183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.847147 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientPID" May 20 21:43:05.848605 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.848591 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 21:43:05.848720 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.848707 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.848761 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.848738 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 21:43:05.849395 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.849379 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientMemory" May 20 21:43:05.849475 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.849410 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasNoDiskPressure" May 20 21:43:05.849475 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.849379 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientMemory" May 20 21:43:05.849475 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.849450 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasNoDiskPressure" May 20 21:43:05.849475 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.849420 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientPID" May 20 21:43:05.849475 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.849467 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientPID" May 20 21:43:05.850642 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.850629 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" May 20 21:43:05.850698 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.850653 2565 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 20 21:43:05.851631 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.851566 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientMemory" May 20 21:43:05.851698 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.851641 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasNoDiskPressure" May 20 21:43:05.851698 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.851652 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeHasSufficientPID" May 20 21:43:05.853733 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.853718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec8abf2a5fcc8cf40a4e9747a451ea23-config\") pod \"kube-apiserver-proxy-ip-10-0-141-67.ec2.internal\" (UID: \"ec8abf2a5fcc8cf40a4e9747a451ea23\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" May 20 21:43:05.853822 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.853739 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0218afb08614bcc2a1af3e601ad7ca82-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal\" (UID: \"0218afb08614bcc2a1af3e601ad7ca82\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.853822 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.853755 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0218afb08614bcc2a1af3e601ad7ca82-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal\" (UID: \"0218afb08614bcc2a1af3e601ad7ca82\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.868080 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.868062 2565 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-67.ec2.internal\" not found" node="ip-10-0-141-67.ec2.internal" May 20 21:43:05.869968 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.869953 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:05.872352 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.872337 2565 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-67.ec2.internal\" not found" node="ip-10-0-141-67.ec2.internal" May 20 21:43:05.954825 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.954796 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0218afb08614bcc2a1af3e601ad7ca82-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal\" (UID: \"0218afb08614bcc2a1af3e601ad7ca82\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.954977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.954832 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec8abf2a5fcc8cf40a4e9747a451ea23-config\") pod \"kube-apiserver-proxy-ip-10-0-141-67.ec2.internal\" (UID: \"ec8abf2a5fcc8cf40a4e9747a451ea23\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" May 20 21:43:05.954977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.954857 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0218afb08614bcc2a1af3e601ad7ca82-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal\" (UID: \"0218afb08614bcc2a1af3e601ad7ca82\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.954977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.954893 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0218afb08614bcc2a1af3e601ad7ca82-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal\" (UID: \"0218afb08614bcc2a1af3e601ad7ca82\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.954977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.954903 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ec8abf2a5fcc8cf40a4e9747a451ea23-config\") pod \"kube-apiserver-proxy-ip-10-0-141-67.ec2.internal\" (UID: \"ec8abf2a5fcc8cf40a4e9747a451ea23\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" May 20 21:43:05.954977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:05.954893 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0218afb08614bcc2a1af3e601ad7ca82-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal\" (UID: \"0218afb08614bcc2a1af3e601ad7ca82\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:05.970949 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:05.970925 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.071022 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.070983 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.170838 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.170769 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:06.171052 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.171038 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.175538 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.175516 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" May 20 21:43:06.271676 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.271634 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.372160 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.372132 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.472797 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.472737 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.562158 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.562133 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 20 21:43:06.562758 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.562311 2565 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 20 21:43:06.573300 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.573276 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.640870 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:06.640840 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0218afb08614bcc2a1af3e601ad7ca82.slice/crio-d8a8fcd206ffecd70b2592176ba2cb48ca6139f0366f03a10ecae6eed1e5f66b WatchSource:0}: Error finding container d8a8fcd206ffecd70b2592176ba2cb48ca6139f0366f03a10ecae6eed1e5f66b: Status 404 returned error can't find the container with id d8a8fcd206ffecd70b2592176ba2cb48ca6139f0366f03a10ecae6eed1e5f66b May 20 21:43:06.641093 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:06.641080 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec8abf2a5fcc8cf40a4e9747a451ea23.slice/crio-ec02aff7d9510e230162c7ce8308ecae6e244a5e2c9af29c33665867db58d8ba WatchSource:0}: Error finding container ec02aff7d9510e230162c7ce8308ecae6e244a5e2c9af29c33665867db58d8ba: Status 404 returned error can't find the container with id ec02aff7d9510e230162c7ce8308ecae6e244a5e2c9af29c33665867db58d8ba May 20 21:43:06.644167 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.644151 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 21:43:06.648134 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.648118 2565 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 20 21:43:06.651509 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.651495 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 20 21:43:06.669194 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.669163 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-19 21:38:05 +0000 UTC" deadline="2028-02-28 15:30:44.047576304 +0000 UTC" May 20 21:43:06.669194 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.669187 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15569h47m37.37839179s" May 20 21:43:06.674223 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:06.674211 2565 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-67.ec2.internal\" not found" May 20 21:43:06.674975 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.674962 2565 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 20 21:43:06.701889 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.701872 2565 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 20 21:43:06.716449 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.716432 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mz4jn" May 20 21:43:06.727015 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.726958 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mz4jn" May 20 21:43:06.744332 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.744289 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" event={"ID":"ec8abf2a5fcc8cf40a4e9747a451ea23","Type":"ContainerStarted","Data":"ec02aff7d9510e230162c7ce8308ecae6e244a5e2c9af29c33665867db58d8ba"} May 20 21:43:06.745299 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.745280 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" event={"ID":"0218afb08614bcc2a1af3e601ad7ca82","Type":"ContainerStarted","Data":"d8a8fcd206ffecd70b2592176ba2cb48ca6139f0366f03a10ecae6eed1e5f66b"} May 20 21:43:06.752453 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.752429 2565 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" May 20 21:43:06.787403 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.787379 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 20 21:43:06.788287 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.788276 2565 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" May 20 21:43:06.798712 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:06.798695 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 20 21:43:07.088250 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.088220 2565 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 20 21:43:07.632092 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.632061 2565 apiserver.go:52] "Watching apiserver" May 20 21:43:07.641360 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.641330 2565 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 20 21:43:07.644159 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.644133 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-l6m8q","kube-system/konnectivity-agent-xzvgm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72","openshift-cluster-node-tuning-operator/tuned-46ndm","openshift-multus/multus-additional-cni-plugins-d2q8g","openshift-network-diagnostics/network-check-target-xj72f","kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal","openshift-dns/node-resolver-47h4c","openshift-image-registry/node-ca-knkhq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal","openshift-multus/multus-9bh8z","openshift-multus/network-metrics-daemon-2slgv","openshift-network-operator/iptables-alerter-r7klv"] May 20 21:43:07.646775 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.646746 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:07.646869 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.646838 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:07.647932 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.647911 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.648096 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.648071 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.649436 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.649414 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.650451 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650434 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xj2th\"" May 20 21:43:07.650548 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650460 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:07.650548 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650487 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 20 21:43:07.650548 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650525 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 20 21:43:07.650747 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650721 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 20 21:43:07.650816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650799 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 20 21:43:07.650922 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.650903 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.651467 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.651451 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mcssb\"" May 20 21:43:07.651553 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.651523 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 20 21:43:07.651658 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.651646 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 20 21:43:07.652426 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.652401 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.653072 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653054 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 20 21:43:07.653162 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653087 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 20 21:43:07.653162 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653104 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f6lpv\"" May 20 21:43:07.653273 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653054 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 20 21:43:07.653273 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653231 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 20 21:43:07.653367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653300 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 20 21:43:07.653367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653353 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xsw9r\"" May 20 21:43:07.653464 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653389 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 20 21:43:07.653464 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653403 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 20 21:43:07.653663 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653647 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 20 21:43:07.653731 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653649 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 20 21:43:07.653784 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.653651 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 20 21:43:07.654329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.654314 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.654730 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.654706 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:07.654822 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.654739 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 20 21:43:07.654822 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.654755 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lrpv8\"" May 20 21:43:07.655801 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.655775 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.656766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.656497 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 20 21:43:07.656766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.656545 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q8ck7\"" May 20 21:43:07.656766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.656654 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 20 21:43:07.657995 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.657975 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9bh8z" May 20 21:43:07.658100 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.658003 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 20 21:43:07.658100 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.658088 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 20 21:43:07.658217 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.658127 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 20 21:43:07.658473 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.658453 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8dnl9\"" May 20 21:43:07.658972 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.658956 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:07.659060 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.659020 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:07.660191 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.660173 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 20 21:43:07.660291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.660191 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q4npj\"" May 20 21:43:07.661107 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.661090 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.663209 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-kubelet\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663221 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-systemd\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.663291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663245 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-run\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.663291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663268 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-var-lib-kubelet\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.663438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-var-lib-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-ovn\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663351 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-system-cni-dir\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.663438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663400 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 20 21:43:07.663438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663396 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgb5g\" (UniqueName: \"kubernetes.io/projected/f1000b0a-e70f-465c-8a26-6e030ed46905-kube-api-access-cgb5g\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.663686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663484 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 20 21:43:07.663686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663588 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kls7q\"" May 20 21:43:07.663686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663594 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-systemd-units\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663628 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-env-overrides\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663653 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.663686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663680 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbsgp\" (UniqueName: \"kubernetes.io/projected/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-kube-api-access-dbsgp\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-tmp\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663792 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-cni-netd\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663863 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovn-node-metrics-cert\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663884 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cnibin\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.663950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663926 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65wkt\" (UniqueName: \"kubernetes.io/projected/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-kube-api-access-65wkt\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.663977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64b82855-f1bb-4887-8207-26154822ff4e-host\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664000 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-etc-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664021 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-log-socket\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664044 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-cni-bin\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664082 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmttb\" (UniqueName: \"kubernetes.io/projected/716ecc43-ddb8-4408-988f-17cf83c4e8a5-kube-api-access-nmttb\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-kubernetes\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysctl-d\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664162 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-slash\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovnkube-script-lib\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664233 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysctl-conf\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664263 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-host\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-tuned\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-systemd\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.664359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664357 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-node-log\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovnkube-config\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-sys\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-lib-modules\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664456 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664490 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1000b0a-e70f-465c-8a26-6e030ed46905-iptables-alerter-script\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64b82855-f1bb-4887-8207-26154822ff4e-serviceca\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmkbc\" (UniqueName: \"kubernetes.io/projected/64b82855-f1bb-4887-8207-26154822ff4e-kube-api-access-nmkbc\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664605 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-run-netns\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-os-release\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-modprobe-d\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664735 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysconfig\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.665110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.664773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1000b0a-e70f-465c-8a26-6e030ed46905-host-slash\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.727539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.727509 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-19 21:38:06 +0000 UTC" deadline="2028-01-04 18:14:18.878125177 +0000 UTC" May 20 21:43:07.727539 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.727532 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14252h31m11.150595144s" May 20 21:43:07.753078 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.753054 2565 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 20 21:43:07.765156 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765130 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-systemd\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765316 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765164 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovnkube-config\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765316 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765189 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-sys\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765316 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765217 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-k8s-cni-cncf-io\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.765316 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765190 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-systemd\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765316 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765293 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-sys\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765331 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-socket-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-sys-fs\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765387 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765413 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nmkbc\" (UniqueName: \"kubernetes.io/projected/64b82855-f1bb-4887-8207-26154822ff4e-kube-api-access-nmkbc\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765460 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765502 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-modprobe-d\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765533 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysconfig\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765584 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765558 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-run\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vchzz\" (UniqueName: \"kubernetes.io/projected/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-kube-api-access-vchzz\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765631 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ae77780-68cf-466d-80a6-8c7fd5053fad-agent-certs\") pod \"konnectivity-agent-xzvgm\" (UID: \"3ae77780-68cf-466d-80a6-8c7fd5053fad\") " pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765654 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysconfig\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765661 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-var-lib-kubelet\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765683 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-run\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765698 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-etc-kubernetes\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765730 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ae77780-68cf-466d-80a6-8c7fd5053fad-konnectivity-ca\") pod \"konnectivity-agent-xzvgm\" (UID: \"3ae77780-68cf-466d-80a6-8c7fd5053fad\") " pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765731 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovnkube-config\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765772 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-modprobe-d\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765780 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-var-lib-kubelet\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765831 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-ovn\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765877 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-ovn\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-conf-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765923 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-tmp-dir\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.765942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765941 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-registration-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765962 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-systemd-units\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765990 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-systemd-units\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.765998 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766025 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-tmp\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766044 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-cni-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766066 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-daemon-config\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766091 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-cni-netd\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766113 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766142 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cnibin\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766167 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-65wkt\" (UniqueName: \"kubernetes.io/projected/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-kube-api-access-65wkt\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766196 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-cni-multus\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766213 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cnibin\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766220 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-hostroot\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766247 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766261 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-cni-netd\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766275 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-etc-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.766565 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766300 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-kubernetes\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766339 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-etc-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766353 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-tuned\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766378 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-slash\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766395 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-kubernetes\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766402 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766416 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovnkube-script-lib\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766439 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-slash\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766440 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysctl-conf\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766476 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz98h\" (UniqueName: \"kubernetes.io/projected/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-kube-api-access-pz98h\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766509 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-node-log\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766533 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-lib-modules\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766554 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1000b0a-e70f-465c-8a26-6e030ed46905-iptables-alerter-script\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766562 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysctl-conf\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766596 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64b82855-f1bb-4887-8207-26154822ff4e-serviceca\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766619 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-run-netns\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766628 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-node-log\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766643 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-os-release\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.767309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766667 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-systemd\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766693 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-system-cni-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766716 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1000b0a-e70f-465c-8a26-6e030ed46905-host-slash\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766738 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-lib-modules\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-kubelet\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766744 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-kubelet\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-run-netns\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766831 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-socket-dir-parent\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766873 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-os-release\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766916 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-var-lib-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766942 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-system-cni-dir\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766958 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1000b0a-e70f-465c-8a26-6e030ed46905-host-slash\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766963 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-cnibin\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766984 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmjs\" (UniqueName: \"kubernetes.io/projected/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-kube-api-access-7hmjs\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766990 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovnkube-script-lib\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.768214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6plp\" (UniqueName: \"kubernetes.io/projected/740a614b-3d7c-4c76-a3ed-162a6819179c-kube-api-access-p6plp\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.766921 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-systemd\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767026 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64b82855-f1bb-4887-8207-26154822ff4e-serviceca\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767045 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cgb5g\" (UniqueName: \"kubernetes.io/projected/f1000b0a-e70f-465c-8a26-6e030ed46905-kube-api-access-cgb5g\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767077 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-env-overrides\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767096 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-var-lib-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767103 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767214 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-system-cni-dir\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767266 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767306 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dbsgp\" (UniqueName: \"kubernetes.io/projected/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-kube-api-access-dbsgp\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-os-release\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767380 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-kubelet\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-multus-certs\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767426 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/716ecc43-ddb8-4408-988f-17cf83c4e8a5-env-overrides\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767427 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767476 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-run-openvswitch\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767479 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767510 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767515 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovn-node-metrics-cert\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767547 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-cni-binary-copy\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-netns\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-hosts-file\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-device-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767709 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64b82855-f1bb-4887-8207-26154822ff4e-host\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767790 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64b82855-f1bb-4887-8207-26154822ff4e-host\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767836 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-log-socket\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767870 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-cni-bin\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767919 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-log-socket\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767971 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nmttb\" (UniqueName: \"kubernetes.io/projected/716ecc43-ddb8-4408-988f-17cf83c4e8a5-kube-api-access-nmttb\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767980 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/716ecc43-ddb8-4408-988f-17cf83c4e8a5-host-cni-bin\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.767991 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768014 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysctl-d\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768125 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-host\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768143 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1000b0a-e70f-465c-8a26-6e030ed46905-iptables-alerter-script\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.769816 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768160 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-cni-bin\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768187 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-sysctl-d\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768203 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-host\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768247 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768280 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-etc-selinux\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768712 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.768796 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.769923 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-etc-tuned\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.770556 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.769949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-tmp\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.770821 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.770766 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/716ecc43-ddb8-4408-988f-17cf83c4e8a5-ovn-node-metrics-cert\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.777639 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.777615 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:07.777639 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.777642 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:07.777798 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.777656 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:07.777798 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.777739 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:08.277705712 +0000 UTC m=+3.059138945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:07.780418 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.780382 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65wkt\" (UniqueName: \"kubernetes.io/projected/8c2f5e5d-3836-47bb-813e-a72e3e9236e6-kube-api-access-65wkt\") pod \"tuned-46ndm\" (UID: \"8c2f5e5d-3836-47bb-813e-a72e3e9236e6\") " pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.780532 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.780396 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmttb\" (UniqueName: \"kubernetes.io/projected/716ecc43-ddb8-4408-988f-17cf83c4e8a5-kube-api-access-nmttb\") pod \"ovnkube-node-l6m8q\" (UID: \"716ecc43-ddb8-4408-988f-17cf83c4e8a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.780719 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.780692 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmkbc\" (UniqueName: \"kubernetes.io/projected/64b82855-f1bb-4887-8207-26154822ff4e-kube-api-access-nmkbc\") pod \"node-ca-knkhq\" (UID: \"64b82855-f1bb-4887-8207-26154822ff4e\") " pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.781219 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.781197 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgb5g\" (UniqueName: \"kubernetes.io/projected/f1000b0a-e70f-465c-8a26-6e030ed46905-kube-api-access-cgb5g\") pod \"iptables-alerter-r7klv\" (UID: \"f1000b0a-e70f-465c-8a26-6e030ed46905\") " pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.781633 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.781609 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbsgp\" (UniqueName: \"kubernetes.io/projected/af93b9bc-b94e-43f9-a980-11e0a7f3d6f6-kube-api-access-dbsgp\") pod \"multus-additional-cni-plugins-d2q8g\" (UID: \"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6\") " pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.869589 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869543 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-registration-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869601 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-cni-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869630 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-daemon-config\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869665 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-cni-multus\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869665 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-registration-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869688 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-hostroot\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869732 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pz98h\" (UniqueName: \"kubernetes.io/projected/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-kube-api-access-pz98h\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869737 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-cni-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869745 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-cni-multus\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.869776 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869762 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-system-cni-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869789 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-socket-dir-parent\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869802 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-hostroot\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869810 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-system-cni-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869835 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-socket-dir-parent\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869905 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869933 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.869959 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-cnibin\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.869992 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870004 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmjs\" (UniqueName: \"kubernetes.io/projected/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-kube-api-access-7hmjs\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870013 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870030 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p6plp\" (UniqueName: \"kubernetes.io/projected/740a614b-3d7c-4c76-a3ed-162a6819179c-kube-api-access-p6plp\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:07.870068 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:08.370050218 +0000 UTC m=+3.151483464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870082 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-cnibin\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870098 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-os-release\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870124 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-kubelet\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870161 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-multus-certs\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.870201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870187 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-cni-binary-copy\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870210 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-netns\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870213 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-os-release\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870223 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-kubelet\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870236 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-hosts-file\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870264 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-netns\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870227 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-multus-certs\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870265 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-device-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870323 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-device-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870339 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-cni-bin\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870366 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-etc-selinux\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870383 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-k8s-cni-cncf-io\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870388 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-hosts-file\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870403 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-socket-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870426 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-sys-fs\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870452 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vchzz\" (UniqueName: \"kubernetes.io/projected/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-kube-api-access-vchzz\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870475 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ae77780-68cf-466d-80a6-8c7fd5053fad-agent-certs\") pod \"konnectivity-agent-xzvgm\" (UID: \"3ae77780-68cf-466d-80a6-8c7fd5053fad\") " pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870500 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-etc-selinux\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871006 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-run-k8s-cni-cncf-io\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870546 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-sys-fs\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870552 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-host-var-lib-cni-bin\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870599 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-etc-kubernetes\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870626 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ae77780-68cf-466d-80a6-8c7fd5053fad-konnectivity-ca\") pod \"konnectivity-agent-xzvgm\" (UID: \"3ae77780-68cf-466d-80a6-8c7fd5053fad\") " pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870725 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-conf-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870762 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-tmp-dir\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870771 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-cni-binary-copy\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870660 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/740a614b-3d7c-4c76-a3ed-162a6819179c-socket-dir\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870691 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-etc-kubernetes\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870875 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-daemon-config\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.870893 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-multus-conf-dir\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.871045 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-tmp-dir\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.871703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.871233 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ae77780-68cf-466d-80a6-8c7fd5053fad-konnectivity-ca\") pod \"konnectivity-agent-xzvgm\" (UID: \"3ae77780-68cf-466d-80a6-8c7fd5053fad\") " pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.873212 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.873190 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ae77780-68cf-466d-80a6-8c7fd5053fad-agent-certs\") pod \"konnectivity-agent-xzvgm\" (UID: \"3ae77780-68cf-466d-80a6-8c7fd5053fad\") " pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:07.877782 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.877755 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz98h\" (UniqueName: \"kubernetes.io/projected/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-kube-api-access-pz98h\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:07.877915 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.877898 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6plp\" (UniqueName: \"kubernetes.io/projected/740a614b-3d7c-4c76-a3ed-162a6819179c-kube-api-access-p6plp\") pod \"aws-ebs-csi-driver-node-tcg72\" (UID: \"740a614b-3d7c-4c76-a3ed-162a6819179c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:07.878163 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.878148 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmjs\" (UniqueName: \"kubernetes.io/projected/c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2-kube-api-access-7hmjs\") pod \"node-resolver-47h4c\" (UID: \"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2\") " pod="openshift-dns/node-resolver-47h4c" May 20 21:43:07.882469 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.882418 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vchzz\" (UniqueName: \"kubernetes.io/projected/325e3bc6-4b21-490f-b8d4-8344c0e5b6c7-kube-api-access-vchzz\") pod \"multus-9bh8z\" (UID: \"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7\") " pod="openshift-multus/multus-9bh8z" May 20 21:43:07.961701 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.961662 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r7klv" May 20 21:43:07.968827 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.968801 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-knkhq" May 20 21:43:07.978128 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.978106 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:07.983198 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.983180 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" May 20 21:43:07.990427 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.990397 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-46ndm" May 20 21:43:07.997139 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:07.997117 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-47h4c" May 20 21:43:08.003165 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.003143 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" May 20 21:43:08.009689 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.009672 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9bh8z" May 20 21:43:08.014235 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.014214 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:08.098812 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.098785 2565 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 20 21:43:08.229389 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.229355 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e8afa1_caeb_4f0c_9d11_2ec5cea5a0d2.slice/crio-12225c07776c3013f609eb3296f2db750c6ca526b832926320a6980504b59ad0 WatchSource:0}: Error finding container 12225c07776c3013f609eb3296f2db750c6ca526b832926320a6980504b59ad0: Status 404 returned error can't find the container with id 12225c07776c3013f609eb3296f2db750c6ca526b832926320a6980504b59ad0 May 20 21:43:08.230852 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.230831 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1000b0a_e70f_465c_8a26_6e030ed46905.slice/crio-04fe23759473ece261520c0097e95485f5c6874fa2cef40db5221765a1f146c8 WatchSource:0}: Error finding container 04fe23759473ece261520c0097e95485f5c6874fa2cef40db5221765a1f146c8: Status 404 returned error can't find the container with id 04fe23759473ece261520c0097e95485f5c6874fa2cef40db5221765a1f146c8 May 20 21:43:08.231440 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.231419 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ae77780_68cf_466d_80a6_8c7fd5053fad.slice/crio-bb0635064323f49fc477ad66adc0e9ff3daf101ac3a9f3f82be3607535f7fb83 WatchSource:0}: Error finding container bb0635064323f49fc477ad66adc0e9ff3daf101ac3a9f3f82be3607535f7fb83: Status 404 returned error can't find the container with id bb0635064323f49fc477ad66adc0e9ff3daf101ac3a9f3f82be3607535f7fb83 May 20 21:43:08.234151 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.234124 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c2f5e5d_3836_47bb_813e_a72e3e9236e6.slice/crio-02143ad7403dba926abdb0fa946b77f52180c92075817734de757f309a70f0da WatchSource:0}: Error finding container 02143ad7403dba926abdb0fa946b77f52180c92075817734de757f309a70f0da: Status 404 returned error can't find the container with id 02143ad7403dba926abdb0fa946b77f52180c92075817734de757f309a70f0da May 20 21:43:08.234903 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.234881 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod325e3bc6_4b21_490f_b8d4_8344c0e5b6c7.slice/crio-a25f0f470c96b2ab819108af7dff5b863feb1122997f23e9e22fb8a15bfaf0e7 WatchSource:0}: Error finding container a25f0f470c96b2ab819108af7dff5b863feb1122997f23e9e22fb8a15bfaf0e7: Status 404 returned error can't find the container with id a25f0f470c96b2ab819108af7dff5b863feb1122997f23e9e22fb8a15bfaf0e7 May 20 21:43:08.236780 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.236500 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod740a614b_3d7c_4c76_a3ed_162a6819179c.slice/crio-4c35c4e0093f4f13ca4c4c2b659050a6acf8d2c5a5093f763e2e08326649c60e WatchSource:0}: Error finding container 4c35c4e0093f4f13ca4c4c2b659050a6acf8d2c5a5093f763e2e08326649c60e: Status 404 returned error can't find the container with id 4c35c4e0093f4f13ca4c4c2b659050a6acf8d2c5a5093f763e2e08326649c60e May 20 21:43:08.237392 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.237328 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf93b9bc_b94e_43f9_a980_11e0a7f3d6f6.slice/crio-754df2fbec8433aef6aaaa3b644a9428ce2848e9257d88c8708e4acdfae5daac WatchSource:0}: Error finding container 754df2fbec8433aef6aaaa3b644a9428ce2848e9257d88c8708e4acdfae5daac: Status 404 returned error can't find the container with id 754df2fbec8433aef6aaaa3b644a9428ce2848e9257d88c8708e4acdfae5daac May 20 21:43:08.267769 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.267738 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716ecc43_ddb8_4408_988f_17cf83c4e8a5.slice/crio-ee1620f38026c55f0f54b8b376b427a854521537da8fd7bce51d7905e648f925 WatchSource:0}: Error finding container ee1620f38026c55f0f54b8b376b427a854521537da8fd7bce51d7905e648f925: Status 404 returned error can't find the container with id ee1620f38026c55f0f54b8b376b427a854521537da8fd7bce51d7905e648f925 May 20 21:43:08.268310 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:08.268287 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b82855_f1bb_4887_8207_26154822ff4e.slice/crio-3f9af06d2444f4f63bcd3cf9b192530cce08bfc16a56a6d0d4625194c72def73 WatchSource:0}: Error finding container 3f9af06d2444f4f63bcd3cf9b192530cce08bfc16a56a6d0d4625194c72def73: Status 404 returned error can't find the container with id 3f9af06d2444f4f63bcd3cf9b192530cce08bfc16a56a6d0d4625194c72def73 May 20 21:43:08.373952 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.373759 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:08.374071 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:08.373895 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:08.374071 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:08.374022 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:08.374071 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:08.374037 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:08.374071 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:08.374054 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:08.374248 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:08.374089 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:09.374070126 +0000 UTC m=+4.155503357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:08.374248 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:08.374105 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:09.374094837 +0000 UTC m=+4.155528067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:08.374248 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.373990 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:08.728504 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.728463 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-19 21:38:06 +0000 UTC" deadline="2028-01-07 02:08:12.650071383 +0000 UTC" May 20 21:43:08.728504 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.728500 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14308h25m3.921575375s" May 20 21:43:08.752641 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.752601 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" event={"ID":"ec8abf2a5fcc8cf40a4e9747a451ea23","Type":"ContainerStarted","Data":"05e69042d46485fc29e233de545e1f520858bf17f295ac670016704f9d69842a"} May 20 21:43:08.765343 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.765267 2565 generic.go:358] "Generic (PLEG): container finished" podID="0218afb08614bcc2a1af3e601ad7ca82" containerID="86f145ecb7c2bb6db72b1c68214ef6860e1c7e6baa4957cb974b5c2175b32cb2" exitCode=0 May 20 21:43:08.765501 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.765343 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" event={"ID":"0218afb08614bcc2a1af3e601ad7ca82","Type":"ContainerDied","Data":"86f145ecb7c2bb6db72b1c68214ef6860e1c7e6baa4957cb974b5c2175b32cb2"} May 20 21:43:08.768001 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.767903 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-knkhq" event={"ID":"64b82855-f1bb-4887-8207-26154822ff4e","Type":"ContainerStarted","Data":"3f9af06d2444f4f63bcd3cf9b192530cce08bfc16a56a6d0d4625194c72def73"} May 20 21:43:08.784877 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.782463 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"ee1620f38026c55f0f54b8b376b427a854521537da8fd7bce51d7905e648f925"} May 20 21:43:08.795770 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.794290 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-67.ec2.internal" podStartSLOduration=2.794271313 podStartE2EDuration="2.794271313s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:43:08.776391391 +0000 UTC m=+3.557824642" watchObservedRunningTime="2026-05-20 21:43:08.794271313 +0000 UTC m=+3.575704565" May 20 21:43:08.810273 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.810010 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" event={"ID":"740a614b-3d7c-4c76-a3ed-162a6819179c","Type":"ContainerStarted","Data":"4c35c4e0093f4f13ca4c4c2b659050a6acf8d2c5a5093f763e2e08326649c60e"} May 20 21:43:08.811619 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.811547 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9bh8z" event={"ID":"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7","Type":"ContainerStarted","Data":"a25f0f470c96b2ab819108af7dff5b863feb1122997f23e9e22fb8a15bfaf0e7"} May 20 21:43:08.848251 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.848200 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46ndm" event={"ID":"8c2f5e5d-3836-47bb-813e-a72e3e9236e6","Type":"ContainerStarted","Data":"02143ad7403dba926abdb0fa946b77f52180c92075817734de757f309a70f0da"} May 20 21:43:08.859683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.859648 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xzvgm" event={"ID":"3ae77780-68cf-466d-80a6-8c7fd5053fad","Type":"ContainerStarted","Data":"bb0635064323f49fc477ad66adc0e9ff3daf101ac3a9f3f82be3607535f7fb83"} May 20 21:43:08.861702 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.861674 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerStarted","Data":"754df2fbec8433aef6aaaa3b644a9428ce2848e9257d88c8708e4acdfae5daac"} May 20 21:43:08.871449 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.871419 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7klv" event={"ID":"f1000b0a-e70f-465c-8a26-6e030ed46905","Type":"ContainerStarted","Data":"04fe23759473ece261520c0097e95485f5c6874fa2cef40db5221765a1f146c8"} May 20 21:43:08.889292 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:08.887921 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-47h4c" event={"ID":"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2","Type":"ContainerStarted","Data":"12225c07776c3013f609eb3296f2db750c6ca526b832926320a6980504b59ad0"} May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:09.381831 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:09.381892 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.382021 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.382024 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.382048 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.382062 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.382081 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:11.382062078 +0000 UTC m=+6.163495313 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:09.382465 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.382113 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:11.382096168 +0000 UTC m=+6.163529402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:09.744396 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:09.744321 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:09.744811 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.744459 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:09.745013 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:09.744974 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:09.745136 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:09.745101 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:09.894700 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:09.894593 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" event={"ID":"0218afb08614bcc2a1af3e601ad7ca82","Type":"ContainerStarted","Data":"ee5a468ffa09ccdba882435c02ff72dc792c66a69d7d600cf958f7eb6798c0e0"} May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:11.397431 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:11.397512 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.397662 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.397680 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.397693 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.397752 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:15.397734234 +0000 UTC m=+10.179167465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.398154 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:11.398287 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.398208 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:15.398193672 +0000 UTC m=+10.179626909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:11.749250 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:11.749222 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:11.749425 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:11.749222 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:11.749425 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.749356 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:11.749543 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:11.749429 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:13.741794 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:13.741752 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:13.742241 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:13.741891 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:13.742525 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:13.742384 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:13.742525 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:13.742490 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:15.430906 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:15.430871 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:15.430930 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.431035 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.431044 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.431050 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.431079 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.431120 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:23.431100169 +0000 UTC m=+18.212533403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:15.431364 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.431140 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:23.431129384 +0000 UTC m=+18.212562619 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:15.742430 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:15.742349 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:15.742597 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.742448 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:15.742597 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:15.742545 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:15.742717 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:15.742670 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:17.742099 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:17.742066 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:17.742099 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:17.742103 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:17.742600 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:17.742183 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:17.742600 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:17.742321 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:19.741709 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:19.741674 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:19.741709 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:19.741708 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:19.742174 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:19.741801 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:19.742174 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:19.741977 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:21.744528 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:21.744494 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:21.744902 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:21.744494 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:21.744902 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:21.744652 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:21.744902 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:21.744770 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:23.493087 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:23.493050 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:23.493108 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.493201 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.493231 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.493242 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.493289 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:39.493274273 +0000 UTC m=+34.274707503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.493204 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:23.493505 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.493341 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:39.493326968 +0000 UTC m=+34.274760198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:23.744347 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:23.744277 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:23.744493 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:23.744277 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:23.744493 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.744390 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:23.744493 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:23.744436 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:25.746266 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:25.745199 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:25.746266 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:25.745310 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:25.746266 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:25.745387 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:25.746266 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:25.745449 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:26.926680 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.926296 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-knkhq" event={"ID":"64b82855-f1bb-4887-8207-26154822ff4e","Type":"ContainerStarted","Data":"3661e4eca73790ac16cdd5db17bf34cc3f824da86ae7cee4696cb775477659fc"} May 20 21:43:26.927703 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.927686 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"e8bba960a2202727e36f403c8b1d6c0475c993460223284466068b24d8559a36"} May 20 21:43:26.927757 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.927713 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"6d66d2ad8bc5e41bc79d820167d07294d1ef08e1e98ea26be09d53b1296022a9"} May 20 21:43:26.928701 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.928686 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" event={"ID":"740a614b-3d7c-4c76-a3ed-162a6819179c","Type":"ContainerStarted","Data":"b3bf03ec759671ac7d6ddae5f0de1c7f6770c7cdfca5791f6db409175ae7e1d7"} May 20 21:43:26.929929 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.929902 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9bh8z" event={"ID":"325e3bc6-4b21-490f-b8d4-8344c0e5b6c7","Type":"ContainerStarted","Data":"584b523de89f481489f136d190dffdfe22e2a08f64ea9f6728e49c30802cce01"} May 20 21:43:26.931283 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.931266 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-46ndm" event={"ID":"8c2f5e5d-3836-47bb-813e-a72e3e9236e6","Type":"ContainerStarted","Data":"96498e69834386e4299d03eb6cdea9289dd7d714879f132c1db037305c47fe4d"} May 20 21:43:26.932399 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.932381 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xzvgm" event={"ID":"3ae77780-68cf-466d-80a6-8c7fd5053fad","Type":"ContainerStarted","Data":"79d8c16a43320a4f3d59dbffda961630c80b0f85bdcb2674edf4746454313c5b"} May 20 21:43:26.933532 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.933512 2565 generic.go:358] "Generic (PLEG): container finished" podID="af93b9bc-b94e-43f9-a980-11e0a7f3d6f6" containerID="62f01a6e42ebe9007ccbb5604adf42a5138d56fc26c6464d043295769c0d2fed" exitCode=0 May 20 21:43:26.933630 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.933560 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerDied","Data":"62f01a6e42ebe9007ccbb5604adf42a5138d56fc26c6464d043295769c0d2fed"} May 20 21:43:26.934803 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.934752 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-47h4c" event={"ID":"c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2","Type":"ContainerStarted","Data":"353ef1e36e4e18e2d8e4c44a88e81e0de727ad85e6381e4002f214c02e37caed"} May 20 21:43:26.942382 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.942342 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-knkhq" podStartSLOduration=7.4549609 podStartE2EDuration="21.942333271s" podCreationTimestamp="2026-05-20 21:43:05 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.269839786 +0000 UTC m=+3.051273019" lastFinishedPulling="2026-05-20 21:43:22.757212154 +0000 UTC m=+17.538645390" observedRunningTime="2026-05-20 21:43:26.942200814 +0000 UTC m=+21.723634066" watchObservedRunningTime="2026-05-20 21:43:26.942333271 +0000 UTC m=+21.723766528" May 20 21:43:26.942458 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.942409 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-67.ec2.internal" podStartSLOduration=20.942406086 podStartE2EDuration="20.942406086s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:43:09.910334183 +0000 UTC m=+4.691767437" watchObservedRunningTime="2026-05-20 21:43:26.942406086 +0000 UTC m=+21.723839336" May 20 21:43:26.983943 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:26.983905 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-46ndm" podStartSLOduration=2.718792578 podStartE2EDuration="20.983891066s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.235784821 +0000 UTC m=+3.017218050" lastFinishedPulling="2026-05-20 21:43:26.500883296 +0000 UTC m=+21.282316538" observedRunningTime="2026-05-20 21:43:26.983415509 +0000 UTC m=+21.764848761" watchObservedRunningTime="2026-05-20 21:43:26.983891066 +0000 UTC m=+21.765324342" May 20 21:43:27.025138 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.025084 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-47h4c" podStartSLOduration=2.7567220519999998 podStartE2EDuration="21.025068244s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.230941642 +0000 UTC m=+3.012374871" lastFinishedPulling="2026-05-20 21:43:26.49928782 +0000 UTC m=+21.280721063" observedRunningTime="2026-05-20 21:43:26.997058212 +0000 UTC m=+21.778491465" watchObservedRunningTime="2026-05-20 21:43:27.025068244 +0000 UTC m=+21.806501501" May 20 21:43:27.025276 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.025249 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-9bh8z" podStartSLOduration=2.7321812100000002 podStartE2EDuration="21.025242635s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.236875492 +0000 UTC m=+3.018308735" lastFinishedPulling="2026-05-20 21:43:26.529936928 +0000 UTC m=+21.311370160" observedRunningTime="2026-05-20 21:43:27.02409637 +0000 UTC m=+21.805529621" watchObservedRunningTime="2026-05-20 21:43:27.025242635 +0000 UTC m=+21.806675889" May 20 21:43:27.046993 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.046888 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xzvgm" podStartSLOduration=6.522819321 podStartE2EDuration="21.046874311s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.233160104 +0000 UTC m=+3.014593349" lastFinishedPulling="2026-05-20 21:43:22.757215095 +0000 UTC m=+17.538648339" observedRunningTime="2026-05-20 21:43:27.046485021 +0000 UTC m=+21.827918265" watchObservedRunningTime="2026-05-20 21:43:27.046874311 +0000 UTC m=+21.828307563" May 20 21:43:27.744471 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.744436 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:27.744682 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.744436 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:27.744682 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:27.744553 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:27.744682 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:27.744639 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:27.938298 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.938242 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r7klv" event={"ID":"f1000b0a-e70f-465c-8a26-6e030ed46905","Type":"ContainerStarted","Data":"467024353aca16f35a9905526343d645f83ca1515f999f9f39366fa3e309e4d0"} May 20 21:43:27.941122 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.941088 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"6ed65b50b8e5c1b1596dcae713414986e5fde902b5040ddce6921baa7a914b77"} May 20 21:43:27.941319 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.941130 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"861629eef7884d4fbd760b4340db8a2ce5f3d38f48d88b0094c4ea7ce3cbc620"} May 20 21:43:27.941319 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.941145 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"2f8aa295ee52b29597a21c726554ca23d425898874e0b0888b7e3e949749b5e5"} May 20 21:43:27.941319 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.941158 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"6c89ddfb2831a94f926c88a527103664a7e5b8134e26635a0f8ec74b6fa57a74"} May 20 21:43:27.953393 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:27.953345 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r7klv" podStartSLOduration=4.687411691 podStartE2EDuration="22.953334352s" podCreationTimestamp="2026-05-20 21:43:05 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.233359936 +0000 UTC m=+3.014793171" lastFinishedPulling="2026-05-20 21:43:26.49928259 +0000 UTC m=+21.280715832" observedRunningTime="2026-05-20 21:43:27.953120346 +0000 UTC m=+22.734553597" watchObservedRunningTime="2026-05-20 21:43:27.953334352 +0000 UTC m=+22.734767604" May 20 21:43:28.139750 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:28.139732 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 20 21:43:28.740838 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:28.740729 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-20T21:43:28.139746761Z","UUID":"4694d723-dad2-41e4-bbde-2093d6ab138a","Handler":null,"Name":"","Endpoint":""} May 20 21:43:28.743017 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:28.742993 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 20 21:43:28.743017 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:28.743023 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 20 21:43:28.945415 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:28.945374 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" event={"ID":"740a614b-3d7c-4c76-a3ed-162a6819179c","Type":"ContainerStarted","Data":"ab418ec772484bc17864ce5681304aee05c52df19e0b0c04469110b959e3e3bb"} May 20 21:43:29.745103 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:29.745012 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:29.745254 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:29.745036 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:29.745254 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:29.745136 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:29.745254 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:29.745231 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:29.949871 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:29.949813 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"a9aa9f62dd39e7a9a40b003a3c8301f9ca4d44164fd2482be04f23242a80d528"} May 20 21:43:29.951397 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:29.951374 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" event={"ID":"740a614b-3d7c-4c76-a3ed-162a6819179c","Type":"ContainerStarted","Data":"a3cdfb50967211c2a26b138cda7b14336bb1cb7ab66474956f86d713d24ff557"} May 20 21:43:29.980104 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:29.979074 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tcg72" podStartSLOduration=2.998436285 podStartE2EDuration="23.979053697s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.238424973 +0000 UTC m=+3.019858202" lastFinishedPulling="2026-05-20 21:43:29.219042381 +0000 UTC m=+24.000475614" observedRunningTime="2026-05-20 21:43:29.977918556 +0000 UTC m=+24.759351789" watchObservedRunningTime="2026-05-20 21:43:29.979053697 +0000 UTC m=+24.760486950" May 20 21:43:31.467679 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:31.467384 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:31.468163 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:31.468145 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:31.742412 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:31.742326 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:31.742564 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:31.742327 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:31.742564 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:31.742460 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:31.742691 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:31.742602 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:31.955093 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:31.955068 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:31.955581 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:31.955551 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xzvgm" May 20 21:43:32.959377 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.959344 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" event={"ID":"716ecc43-ddb8-4408-988f-17cf83c4e8a5","Type":"ContainerStarted","Data":"9e5d7620bff00dae42aee40ec117d48cdd7863ad5fa8bacf4c0ca899d834ad6e"} May 20 21:43:32.960187 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.959661 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:32.960187 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.959686 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:32.960883 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.960678 2565 generic.go:358] "Generic (PLEG): container finished" podID="af93b9bc-b94e-43f9-a980-11e0a7f3d6f6" containerID="c5ba096e5b495776b941cfe999064fd52d4f57355787deec17bb5a1dca272d2f" exitCode=0 May 20 21:43:32.960970 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.960754 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerDied","Data":"c5ba096e5b495776b941cfe999064fd52d4f57355787deec17bb5a1dca272d2f"} May 20 21:43:32.975205 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.975182 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:32.985751 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:32.985709 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" podStartSLOduration=9.492275191 podStartE2EDuration="27.985697053s" podCreationTimestamp="2026-05-20 21:43:05 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.2693924 +0000 UTC m=+3.050825644" lastFinishedPulling="2026-05-20 21:43:26.762814277 +0000 UTC m=+21.544247506" observedRunningTime="2026-05-20 21:43:32.985211433 +0000 UTC m=+27.766644684" watchObservedRunningTime="2026-05-20 21:43:32.985697053 +0000 UTC m=+27.767130305" May 20 21:43:33.742163 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.742132 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:33.742365 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.742132 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:33.742365 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:33.742256 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:33.742365 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:33.742317 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:33.923783 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.923747 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xj72f"] May 20 21:43:33.926654 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.926632 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2slgv"] May 20 21:43:33.964710 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.964675 2565 generic.go:358] "Generic (PLEG): container finished" podID="af93b9bc-b94e-43f9-a980-11e0a7f3d6f6" containerID="096ad24eed37dcc8a56c5a6fc9602344fe20ba5d13f9c63944e7d91b1f4b5636" exitCode=0 May 20 21:43:33.965147 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.964863 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerDied","Data":"096ad24eed37dcc8a56c5a6fc9602344fe20ba5d13f9c63944e7d91b1f4b5636"} May 20 21:43:33.965147 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.964975 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:33.965252 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.965217 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:33.965882 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:33.965616 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:33.965882 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.965694 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:33.965882 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:33.965688 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:33.980190 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:33.980168 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:43:34.968088 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:34.968060 2565 generic.go:358] "Generic (PLEG): container finished" podID="af93b9bc-b94e-43f9-a980-11e0a7f3d6f6" containerID="3bb3cb81835edcc99eb6305a56449b803bce6d3a3c32786fabc7357c4900334b" exitCode=0 May 20 21:43:34.968397 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:34.968109 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerDied","Data":"3bb3cb81835edcc99eb6305a56449b803bce6d3a3c32786fabc7357c4900334b"} May 20 21:43:35.743399 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:35.743363 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:35.743543 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:35.743479 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:35.743543 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:35.743524 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:35.743644 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:35.743628 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:37.741804 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:37.741776 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:37.742326 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:37.741778 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:37.742326 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:37.741898 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:37.742326 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:37.741974 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:39.520027 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:39.519986 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:39.520027 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:39.520042 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:39.520643 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.520156 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 20 21:43:39.520643 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.520185 2565 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 20 21:43:39.520643 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.520200 2565 projected.go:194] Error preparing data for projected volume kube-api-access-wxkjz for pod openshift-network-diagnostics/network-check-target-xj72f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:39.520643 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.520161 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:39.520643 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.520251 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz podName:ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:11.520233489 +0000 UTC m=+66.301666719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wxkjz" (UniqueName: "kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz") pod "network-check-target-xj72f" (UID: "ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 20 21:43:39.520643 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.520324 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:11.52029985 +0000 UTC m=+66.301733094 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : object "openshift-multus"/"metrics-daemon-secret" not registered May 20 21:43:39.741759 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:39.741725 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:39.741945 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:39.741725 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:39.741945 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.741857 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xj72f" podUID="ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5" May 20 21:43:39.741945 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:39.741914 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2slgv" podUID="a5df8c31-3ae1-4ae3-9a87-3da35016fd6f" May 20 21:43:40.486765 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.486737 2565 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-67.ec2.internal" event="NodeReady" May 20 21:43:40.486945 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.486890 2565 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 20 21:43:40.522026 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.521993 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d49b96d4b-jdts2"] May 20 21:43:40.554798 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.554765 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt"] May 20 21:43:40.554963 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.554927 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.558359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.558337 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 20 21:43:40.558359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.558349 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 20 21:43:40.558853 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.558830 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7vrzg\"" May 20 21:43:40.558853 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.558846 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 20 21:43:40.568795 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.568760 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm"] May 20 21:43:40.568909 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.568890 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 20 21:43:40.568968 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.568917 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.572332 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.572314 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-dshgh\"" May 20 21:43:40.572433 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.572316 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 20 21:43:40.572501 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.572489 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" May 20 21:43:40.572559 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.572495 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 20 21:43:40.572655 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.572634 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 20 21:43:40.586671 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.586649 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq"] May 20 21:43:40.586812 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.586795 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.592149 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.592130 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 20 21:43:40.592406 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.592388 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 20 21:43:40.592492 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.592446 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 20 21:43:40.592886 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.592867 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 20 21:43:40.613082 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.613053 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6xhk7"] May 20 21:43:40.613204 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.613189 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.616511 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.616486 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" May 20 21:43:40.630736 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.630720 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6hfxx"] May 20 21:43:40.630915 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.630884 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.633984 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.633964 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ttsl8\"" May 20 21:43:40.634097 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.633973 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 20 21:43:40.637276 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.637259 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 20 21:43:40.645866 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.645844 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d49b96d4b-jdts2"] May 20 21:43:40.645971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.645873 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq"] May 20 21:43:40.645971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.645888 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm"] May 20 21:43:40.645971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.645902 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt"] May 20 21:43:40.645971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.645912 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xhk7"] May 20 21:43:40.646120 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.645994 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:40.646120 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.646008 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hfxx"] May 20 21:43:40.654729 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.654714 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 20 21:43:40.656720 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.656700 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 20 21:43:40.656720 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.656711 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 20 21:43:40.659604 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.659589 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wdrst\"" May 20 21:43:40.728315 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-hub\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-tmp-dir\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728374 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728404 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-image-registry-private-configuration\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9j5t\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-kube-api-access-d9j5t\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728451 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-ca\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728477 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/33defec2-3d29-4850-b9b7-fafe27a4225a-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.728505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728498 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-tmp\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728523 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-klusterlet-config\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/304381e9-3e40-4fc2-93a1-99dac3c2edb0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b676cdb98-rf2qt\" (UID: \"304381e9-3e40-4fc2-93a1-99dac3c2edb0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728591 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728631 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8zpp\" (UniqueName: \"kubernetes.io/projected/33defec2-3d29-4850-b9b7-fafe27a4225a-kube-api-access-d8zpp\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gnk9\" (UniqueName: \"kubernetes.io/projected/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-kube-api-access-2gnk9\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-bound-sa-token\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qg7w\" (UniqueName: \"kubernetes.io/projected/304381e9-3e40-4fc2-93a1-99dac3c2edb0-kube-api-access-2qg7w\") pod \"managed-serviceaccount-addon-agent-b676cdb98-rf2qt\" (UID: \"304381e9-3e40-4fc2-93a1-99dac3c2edb0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728784 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-trusted-ca\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728828 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-ca-trust-extracted\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.728872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728857 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdjjq\" (UniqueName: \"kubernetes.io/projected/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-kube-api-access-rdjjq\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.729389 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.729389 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-installation-pull-secrets\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.729389 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728954 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-config-volume\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.729389 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.728969 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-certificates\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.829505 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829477 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-image-registry-private-configuration\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.829683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829513 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d9j5t\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-kube-api-access-d9j5t\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.829683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829532 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-ca\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.829683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829550 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/33defec2-3d29-4850-b9b7-fafe27a4225a-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.829683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829616 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-tmp\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.829683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829649 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-klusterlet-config\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.829936 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829770 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/304381e9-3e40-4fc2-93a1-99dac3c2edb0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b676cdb98-rf2qt\" (UID: \"304381e9-3e40-4fc2-93a1-99dac3c2edb0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.829936 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829810 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.829936 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829843 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d8zpp\" (UniqueName: \"kubernetes.io/projected/33defec2-3d29-4850-b9b7-fafe27a4225a-kube-api-access-d8zpp\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.829936 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829868 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2gnk9\" (UniqueName: \"kubernetes.io/projected/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-kube-api-access-2gnk9\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.829936 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829898 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.829936 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtgh5\" (UniqueName: \"kubernetes.io/projected/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-kube-api-access-mtgh5\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:40.830224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:40.830224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.829987 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-bound-sa-token\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.830224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830027 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2qg7w\" (UniqueName: \"kubernetes.io/projected/304381e9-3e40-4fc2-93a1-99dac3c2edb0-kube-api-access-2qg7w\") pod \"managed-serviceaccount-addon-agent-b676cdb98-rf2qt\" (UID: \"304381e9-3e40-4fc2-93a1-99dac3c2edb0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.830224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830053 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-trusted-ca\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.830224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830055 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-tmp\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.830449 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830084 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-ca-trust-extracted\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.830449 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830413 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rdjjq\" (UniqueName: \"kubernetes.io/projected/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-kube-api-access-rdjjq\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.830546 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830490 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.830546 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830517 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-installation-pull-secrets\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.830667 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830548 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-config-volume\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.830667 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830612 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-certificates\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.830667 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830630 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/33defec2-3d29-4850-b9b7-fafe27a4225a-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.830667 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830635 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-hub\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.830848 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830710 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-tmp-dir\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.830848 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.830749 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.830848 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.830810 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:43:40.830980 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.830864 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:41.330846073 +0000 UTC m=+36.112279318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:43:40.831119 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.831090 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-ca-trust-extracted\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.831349 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.831325 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-tmp-dir\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.831471 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.831429 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-trusted-ca\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.832504 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.832054 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-certificates\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.832504 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.832291 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:43:40.832504 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.832306 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:43:40.832504 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.832356 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:41.332338213 +0000 UTC m=+36.113771451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:43:40.832504 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.832424 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-config-volume\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.834928 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.834903 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-ca\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.835032 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.834929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-hub\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.835032 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.834937 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.835032 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.834996 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/33defec2-3d29-4850-b9b7-fafe27a4225a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.835174 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.835099 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-klusterlet-config\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.835174 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.835137 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-installation-pull-secrets\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.835248 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.835178 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-image-registry-private-configuration\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.835601 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.835565 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/304381e9-3e40-4fc2-93a1-99dac3c2edb0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-b676cdb98-rf2qt\" (UID: \"304381e9-3e40-4fc2-93a1-99dac3c2edb0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.839802 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.839754 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gnk9\" (UniqueName: \"kubernetes.io/projected/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-kube-api-access-2gnk9\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:40.840081 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.840062 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9j5t\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-kube-api-access-d9j5t\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.840358 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.840329 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qg7w\" (UniqueName: \"kubernetes.io/projected/304381e9-3e40-4fc2-93a1-99dac3c2edb0-kube-api-access-2qg7w\") pod \"managed-serviceaccount-addon-agent-b676cdb98-rf2qt\" (UID: \"304381e9-3e40-4fc2-93a1-99dac3c2edb0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.840562 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.840545 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdjjq\" (UniqueName: \"kubernetes.io/projected/d25ce7f2-161e-4bd1-a968-b84e041d0d2d-kube-api-access-rdjjq\") pod \"klusterlet-addon-workmgr-5dd4b7b474-85sdq\" (UID: \"d25ce7f2-161e-4bd1-a968-b84e041d0d2d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.840700 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.840641 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-bound-sa-token\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:40.841038 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.841004 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8zpp\" (UniqueName: \"kubernetes.io/projected/33defec2-3d29-4850-b9b7-fafe27a4225a-kube-api-access-d8zpp\") pod \"cluster-proxy-proxy-agent-6f9c99bc8c-498wm\" (UID: \"33defec2-3d29-4850-b9b7-fafe27a4225a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.892484 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.892458 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" May 20 21:43:40.900228 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.900125 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:43:40.932134 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.932098 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mtgh5\" (UniqueName: \"kubernetes.io/projected/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-kube-api-access-mtgh5\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:40.932343 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.932139 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:40.932343 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.932253 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:43:40.932343 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:40.932318 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:43:41.432299945 +0000 UTC m=+36.213733177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:43:40.939764 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.939485 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:40.940670 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:40.940634 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtgh5\" (UniqueName: \"kubernetes.io/projected/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-kube-api-access-mtgh5\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:41.077158 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.077129 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt"] May 20 21:43:41.079728 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.079683 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm"] May 20 21:43:41.087066 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.087044 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq"] May 20 21:43:41.127957 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:41.127925 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod304381e9_3e40_4fc2_93a1_99dac3c2edb0.slice/crio-19b572d56a0ebd2cfbb0af623e272e238574cf697ae8546e6dc1b8160ce5470a WatchSource:0}: Error finding container 19b572d56a0ebd2cfbb0af623e272e238574cf697ae8546e6dc1b8160ce5470a: Status 404 returned error can't find the container with id 19b572d56a0ebd2cfbb0af623e272e238574cf697ae8546e6dc1b8160ce5470a May 20 21:43:41.128465 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:41.128437 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33defec2_3d29_4850_b9b7_fafe27a4225a.slice/crio-e7f7555e251420e50f13b01773f5bc2f5265faa018ec2225103d43756b6bbd7a WatchSource:0}: Error finding container e7f7555e251420e50f13b01773f5bc2f5265faa018ec2225103d43756b6bbd7a: Status 404 returned error can't find the container with id e7f7555e251420e50f13b01773f5bc2f5265faa018ec2225103d43756b6bbd7a May 20 21:43:41.129124 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:41.129098 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25ce7f2_161e_4bd1_a968_b84e041d0d2d.slice/crio-2e5563dffefe8cc87777e8912ccccd2f4e4884411fc0902f0d028dc1d06804ec WatchSource:0}: Error finding container 2e5563dffefe8cc87777e8912ccccd2f4e4884411fc0902f0d028dc1d06804ec: Status 404 returned error can't find the container with id 2e5563dffefe8cc87777e8912ccccd2f4e4884411fc0902f0d028dc1d06804ec May 20 21:43:41.336414 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.336320 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:41.336596 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.336502 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:43:41.336596 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.336525 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:43:41.336596 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.336539 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:41.336757 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.336611 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:42.33658974 +0000 UTC m=+37.118022984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:43:41.336757 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.336660 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:43:41.336757 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.336703 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:42.336692879 +0000 UTC m=+37.118126108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:43:41.437187 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.437160 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:41.437324 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.437308 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:43:41.437375 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:41.437367 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:43:42.437352154 +0000 UTC m=+37.218785388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:43:41.741983 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.741901 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:43:41.742864 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.742105 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:43:41.746360 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.746341 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 20 21:43:41.746451 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.746342 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgd7q\"" May 20 21:43:41.746451 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.746364 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 20 21:43:41.746791 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.746774 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8dpp\"" May 20 21:43:41.746840 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.746799 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 20 21:43:41.989524 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.989176 2565 generic.go:358] "Generic (PLEG): container finished" podID="af93b9bc-b94e-43f9-a980-11e0a7f3d6f6" containerID="a7f125e6f506e7cde60e3d8a2d1751dc89e3487235edec36e6e3ee2237c84a9c" exitCode=0 May 20 21:43:41.989524 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.989347 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerDied","Data":"a7f125e6f506e7cde60e3d8a2d1751dc89e3487235edec36e6e3ee2237c84a9c"} May 20 21:43:41.992153 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.992083 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" event={"ID":"d25ce7f2-161e-4bd1-a968-b84e041d0d2d","Type":"ContainerStarted","Data":"2e5563dffefe8cc87777e8912ccccd2f4e4884411fc0902f0d028dc1d06804ec"} May 20 21:43:41.998616 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:41.998592 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" event={"ID":"33defec2-3d29-4850-b9b7-fafe27a4225a","Type":"ContainerStarted","Data":"e7f7555e251420e50f13b01773f5bc2f5265faa018ec2225103d43756b6bbd7a"} May 20 21:43:42.000426 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:42.000378 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" event={"ID":"304381e9-3e40-4fc2-93a1-99dac3c2edb0","Type":"ContainerStarted","Data":"19b572d56a0ebd2cfbb0af623e272e238574cf697ae8546e6dc1b8160ce5470a"} May 20 21:43:42.348151 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:42.348111 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:42.348332 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:42.348213 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:42.348430 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.348409 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:43:42.348483 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.348435 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:43:42.348535 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.348498 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:44.348479807 +0000 UTC m=+39.129913056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:43:42.349506 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.349486 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:43:42.349595 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.349556 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:44.349539554 +0000 UTC m=+39.130972800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:43:42.449416 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:42.448745 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:42.449416 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.448965 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:43:42.449416 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:42.449069 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:43:44.449038525 +0000 UTC m=+39.230471757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:43:43.013138 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:43.012724 2565 generic.go:358] "Generic (PLEG): container finished" podID="af93b9bc-b94e-43f9-a980-11e0a7f3d6f6" containerID="16f00e950909e5039eb992b3d716f53c5c6224b88d411049c46463df1860bd01" exitCode=0 May 20 21:43:43.013138 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:43.012893 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerDied","Data":"16f00e950909e5039eb992b3d716f53c5c6224b88d411049c46463df1860bd01"} May 20 21:43:44.365356 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:44.365322 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:44.365764 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:44.365416 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:44.365764 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.365465 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:43:44.365764 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.365547 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:48.365526108 +0000 UTC m=+43.146959338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:43:44.365764 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.365553 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:43:44.365764 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.365586 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:43:44.365764 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.365634 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:48.365622234 +0000 UTC m=+43.147055464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:43:44.466783 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:44.466744 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:44.466988 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.466890 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:43:44.466988 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:44.466959 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:43:48.466939721 +0000 UTC m=+43.248372969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:43:48.398683 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:48.398645 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:48.399053 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:48.398707 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:48.399053 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.398795 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:43:48.399053 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.398815 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:43:48.399053 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.398863 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:56.398848914 +0000 UTC m=+51.180282148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:43:48.399053 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.398794 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:43:48.399053 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.398935 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:56.398921536 +0000 UTC m=+51.180354765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:43:48.502330 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:48.502305 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:48.502427 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.502417 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:43:48.502477 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:48.502468 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:43:56.502455238 +0000 UTC m=+51.283888467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:43:49.021291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.021258 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-txkmq"] May 20 21:43:49.044069 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.044033 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" event={"ID":"af93b9bc-b94e-43f9-a980-11e0a7f3d6f6","Type":"ContainerStarted","Data":"35b0f15de113b458ddd213bfa3c408420f0b901c34ff6a42e43fb9f7aba1e4ea"} May 20 21:43:49.044069 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.044069 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" event={"ID":"d25ce7f2-161e-4bd1-a968-b84e041d0d2d","Type":"ContainerStarted","Data":"1164ae63a19eed59661d7bf7c087b74a7270c921d2b19ed8eaa57127dc178253"} May 20 21:43:49.044252 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.044083 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" event={"ID":"33defec2-3d29-4850-b9b7-fafe27a4225a","Type":"ContainerStarted","Data":"6883307526c2c05a91d8683fc101a41494c339aec3e43a55f14a77b9a333048d"} May 20 21:43:49.044252 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.044099 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" event={"ID":"304381e9-3e40-4fc2-93a1-99dac3c2edb0","Type":"ContainerStarted","Data":"ac3e77cf3234a46adff28636fcfeb1e66e98c9abeacb0054962b0d0cb7f5873a"} May 20 21:43:49.044252 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.044115 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txkmq"] May 20 21:43:49.044387 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.044364 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.046843 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.046825 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 20 21:43:49.079168 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.079123 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d2q8g" podStartSLOduration=11.160581388 podStartE2EDuration="44.079109793s" podCreationTimestamp="2026-05-20 21:43:05 +0000 UTC" firstStartedPulling="2026-05-20 21:43:08.239146558 +0000 UTC m=+3.020579791" lastFinishedPulling="2026-05-20 21:43:41.157674967 +0000 UTC m=+35.939108196" observedRunningTime="2026-05-20 21:43:49.077957537 +0000 UTC m=+43.859390788" watchObservedRunningTime="2026-05-20 21:43:49.079109793 +0000 UTC m=+43.860543036" May 20 21:43:49.108079 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.108037 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-b676cdb98-rf2qt" podStartSLOduration=5.78628115 podStartE2EDuration="13.108028195s" podCreationTimestamp="2026-05-20 21:43:36 +0000 UTC" firstStartedPulling="2026-05-20 21:43:41.132953865 +0000 UTC m=+35.914387098" lastFinishedPulling="2026-05-20 21:43:48.454700913 +0000 UTC m=+43.236134143" observedRunningTime="2026-05-20 21:43:49.107442111 +0000 UTC m=+43.888875363" watchObservedRunningTime="2026-05-20 21:43:49.108028195 +0000 UTC m=+43.889461440" May 20 21:43:49.127114 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.127076 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" podStartSLOduration=5.754363094 podStartE2EDuration="13.127064592s" podCreationTimestamp="2026-05-20 21:43:36 +0000 UTC" firstStartedPulling="2026-05-20 21:43:41.132846945 +0000 UTC m=+35.914280177" lastFinishedPulling="2026-05-20 21:43:48.505548445 +0000 UTC m=+43.286981675" observedRunningTime="2026-05-20 21:43:49.126654463 +0000 UTC m=+43.908087713" watchObservedRunningTime="2026-05-20 21:43:49.127064592 +0000 UTC m=+43.908497843" May 20 21:43:49.207889 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.207850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-kubelet-config\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.208072 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.207901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-original-pull-secret\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.208072 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.207977 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-dbus\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.308381 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.308350 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-dbus\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.308551 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.308530 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-kubelet-config\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.308642 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.308566 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-original-pull-secret\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.308642 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.308634 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-kubelet-config\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.308727 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.308659 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-dbus\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.311770 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.311745 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7e070039-fa4d-45ae-8c0a-7c5abaa23e35-original-pull-secret\") pod \"global-pull-secret-syncer-txkmq\" (UID: \"7e070039-fa4d-45ae-8c0a-7c5abaa23e35\") " pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.351911 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.351886 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-txkmq" May 20 21:43:49.465285 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:49.465257 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-txkmq"] May 20 21:43:49.468513 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:49.468480 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e070039_fa4d_45ae_8c0a_7c5abaa23e35.slice/crio-e3e05c73721b6c994fcc0f78e84c72b9f877d45f348e12aaf12c3bdef0c59ede WatchSource:0}: Error finding container e3e05c73721b6c994fcc0f78e84c72b9f877d45f348e12aaf12c3bdef0c59ede: Status 404 returned error can't find the container with id e3e05c73721b6c994fcc0f78e84c72b9f877d45f348e12aaf12c3bdef0c59ede May 20 21:43:50.032332 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:50.032278 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txkmq" event={"ID":"7e070039-fa4d-45ae-8c0a-7c5abaa23e35","Type":"ContainerStarted","Data":"e3e05c73721b6c994fcc0f78e84c72b9f877d45f348e12aaf12c3bdef0c59ede"} May 20 21:43:50.032879 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:50.032853 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:50.034425 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:50.034400 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5dd4b7b474-85sdq" May 20 21:43:52.040352 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:52.040315 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" event={"ID":"33defec2-3d29-4850-b9b7-fafe27a4225a","Type":"ContainerStarted","Data":"986a17bffef8d6231538ba57589a9dda6db0fd7b59af0302cfc41cf0049974a9"} May 20 21:43:52.040740 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:52.040361 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" event={"ID":"33defec2-3d29-4850-b9b7-fafe27a4225a","Type":"ContainerStarted","Data":"201b39e3ca4d2e7048b8be1a09838999993f5e1a648f55e889cbecbd354a1d96"} May 20 21:43:52.063965 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:52.063923 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" podStartSLOduration=5.776313854 podStartE2EDuration="16.063909447s" podCreationTimestamp="2026-05-20 21:43:36 +0000 UTC" firstStartedPulling="2026-05-20 21:43:41.133119764 +0000 UTC m=+35.914552997" lastFinishedPulling="2026-05-20 21:43:51.420715361 +0000 UTC m=+46.202148590" observedRunningTime="2026-05-20 21:43:52.063631151 +0000 UTC m=+46.845064414" watchObservedRunningTime="2026-05-20 21:43:52.063909447 +0000 UTC m=+46.845342697" May 20 21:43:54.045275 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:54.045194 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-txkmq" event={"ID":"7e070039-fa4d-45ae-8c0a-7c5abaa23e35","Type":"ContainerStarted","Data":"d489d6a223ec4a01335e15470f0502fc0059b74fdfe8db243f8401106f367ce3"} May 20 21:43:54.060419 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:54.060374 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-txkmq" podStartSLOduration=1.73491106 podStartE2EDuration="6.060361395s" podCreationTimestamp="2026-05-20 21:43:48 +0000 UTC" firstStartedPulling="2026-05-20 21:43:49.470192192 +0000 UTC m=+44.251625423" lastFinishedPulling="2026-05-20 21:43:53.795642513 +0000 UTC m=+48.577075758" observedRunningTime="2026-05-20 21:43:54.059723767 +0000 UTC m=+48.841157017" watchObservedRunningTime="2026-05-20 21:43:54.060361395 +0000 UTC m=+48.841794645" May 20 21:43:56.467787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:56.467746 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:43:56.468232 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:56.467817 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:43:56.468232 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.467902 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:43:56.468232 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.467927 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:43:56.468232 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.467905 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:43:56.468232 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.467981 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:12.467967437 +0000 UTC m=+67.249400666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:43:56.468232 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.467994 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:12.467988376 +0000 UTC m=+67.249421605 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:43:56.568620 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:56.568585 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:43:56.568763 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.568677 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:43:56.568763 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:56.568729 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:44:12.568717694 +0000 UTC m=+67.350150923 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:43:58.060701 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.060660 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf"] May 20 21:43:58.063082 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.063063 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5"] May 20 21:43:58.063225 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.063206 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.064815 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.064797 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-44f4f"] May 20 21:43:58.064941 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.064922 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" May 20 21:43:58.066544 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.066527 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp"] May 20 21:43:58.066677 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.066661 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.067815 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.067796 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" May 20 21:43:58.068283 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.068263 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v"] May 20 21:43:58.068501 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.068388 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.068501 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.068474 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:58.068501 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.068488 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hjt5b\"" May 20 21:43:58.068686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.068625 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" May 20 21:43:58.069544 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.069529 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 20 21:43:58.070503 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.070352 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.072181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.070969 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" May 20 21:43:58.072181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.071043 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" May 20 21:43:58.072181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.071600 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" May 20 21:43:58.072181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.071875 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.073425 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.073794 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.074043 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-gxt4l\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.074090 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.074189 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.074522 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:58.074787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.074596 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-bw8rv\"" May 20 21:43:58.075405 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.075240 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" May 20 21:43:58.075405 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.075388 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" May 20 21:43:58.075695 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.075675 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7gm98\"" May 20 21:43:58.075788 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.075716 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" May 20 21:43:58.075788 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.075776 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-68w58\"" May 20 21:43:58.078126 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.078107 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 20 21:43:58.079437 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46a4c312-92f4-4a59-8acd-96a2c204d00c-config\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.079517 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46a4c312-92f4-4a59-8acd-96a2c204d00c-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.079517 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079475 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/291a4427-81e3-47f4-b3cf-26ce3d5911b7-trusted-ca\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.079517 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf7pm\" (UniqueName: \"kubernetes.io/projected/75d3929a-ac6f-4766-9496-9ae01f5ed769-kube-api-access-mf7pm\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.079659 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079557 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.079659 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079600 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/291a4427-81e3-47f4-b3cf-26ce3d5911b7-config\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.079659 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079620 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.079772 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw9w4\" (UniqueName: \"kubernetes.io/projected/cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad-kube-api-access-dw9w4\") pod \"volume-data-source-validator-5cf8c89d64-wcwl5\" (UID: \"cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" May 20 21:43:58.079772 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079754 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/291a4427-81e3-47f4-b3cf-26ce3d5911b7-serving-cert\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.079870 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw6dc\" (UniqueName: \"kubernetes.io/projected/291a4427-81e3-47f4-b3cf-26ce3d5911b7-kube-api-access-bw6dc\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.079870 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.079965 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079870 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkn2b\" (UniqueName: \"kubernetes.io/projected/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-kube-api-access-kkn2b\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.079965 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.079903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzfg9\" (UniqueName: \"kubernetes.io/projected/46a4c312-92f4-4a59-8acd-96a2c204d00c-kube-api-access-qzfg9\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.080226 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.080208 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" May 20 21:43:58.091825 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.091804 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5"] May 20 21:43:58.093032 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.092993 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v"] May 20 21:43:58.094112 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.094084 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp"] May 20 21:43:58.095188 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.095171 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-44f4f"] May 20 21:43:58.095979 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.095960 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf"] May 20 21:43:58.169598 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.169550 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48"] May 20 21:43:58.171759 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.171743 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p"] May 20 21:43:58.171893 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.171876 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.173478 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.173458 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-kxsw4"] May 20 21:43:58.173607 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.173592 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.174465 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.174447 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fx8b2\"" May 20 21:43:58.174547 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.174485 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" May 20 21:43:58.174687 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.174673 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" May 20 21:43:58.175597 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.175566 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.178264 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178245 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" May 20 21:43:58.178343 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178282 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" May 20 21:43:58.178343 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178283 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" May 20 21:43:58.178464 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178446 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 20 21:43:58.178536 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178473 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" May 20 21:43:58.178860 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178705 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4sv7j\"" May 20 21:43:58.178860 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.178764 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 20 21:43:58.179103 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.179086 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" May 20 21:43:58.179181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.179101 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l9flm\"" May 20 21:43:58.179181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.179157 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" May 20 21:43:58.180260 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180239 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mf7pm\" (UniqueName: \"kubernetes.io/projected/75d3929a-ac6f-4766-9496-9ae01f5ed769-kube-api-access-mf7pm\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.180312 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180285 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.180347 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180318 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/291a4427-81e3-47f4-b3cf-26ce3d5911b7-config\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.180379 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180350 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.180420 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180407 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/55be9913-a333-4eef-90a1-c2b3311cf9a6-snapshots\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.180461 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55be9913-a333-4eef-90a1-c2b3311cf9a6-serving-cert\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.180509 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180466 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zqc4\" (UniqueName: \"kubernetes.io/projected/55be9913-a333-4eef-90a1-c2b3311cf9a6-kube-api-access-5zqc4\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.180558 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180507 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dw9w4\" (UniqueName: \"kubernetes.io/projected/cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad-kube-api-access-dw9w4\") pod \"volume-data-source-validator-5cf8c89d64-wcwl5\" (UID: \"cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" May 20 21:43:58.180558 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180534 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09b71b40-b58b-4831-ac46-0f4c09f6f287-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.180558 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.180537 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 20 21:43:58.180699 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.180617 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls podName:7eb15ff5-c1cf-40e2-a612-7e5494cc395e nodeName:}" failed. No retries permitted until 2026-05-20 21:43:58.680599935 +0000 UTC m=+53.462033178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-72ztf" (UID: "7eb15ff5-c1cf-40e2-a612-7e5494cc395e") : secret "cluster-monitoring-operator-tls" not found May 20 21:43:58.180760 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180691 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/291a4427-81e3-47f4-b3cf-26ce3d5911b7-serving-cert\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.180760 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180726 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55be9913-a333-4eef-90a1-c2b3311cf9a6-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.180854 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180764 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bw6dc\" (UniqueName: \"kubernetes.io/projected/291a4427-81e3-47f4-b3cf-26ce3d5911b7-kube-api-access-bw6dc\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.180854 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180808 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.180951 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180846 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kkn2b\" (UniqueName: \"kubernetes.io/projected/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-kube-api-access-kkn2b\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.180951 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180885 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0260fbc2-244d-4d26-80f6-55febb670445-config\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.180951 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180922 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qzfg9\" (UniqueName: \"kubernetes.io/projected/46a4c312-92f4-4a59-8acd-96a2c204d00c-kube-api-access-qzfg9\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.180966 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55be9913-a333-4eef-90a1-c2b3311cf9a6-tmp\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181005 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46a4c312-92f4-4a59-8acd-96a2c204d00c-config\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.180890 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181068 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46a4c312-92f4-4a59-8acd-96a2c204d00c-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181086 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181099 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/291a4427-81e3-47f4-b3cf-26ce3d5911b7-config\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.181117 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls podName:75d3929a-ac6f-4766-9496-9ae01f5ed769 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:58.681100332 +0000 UTC m=+53.462533562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls") pod "cluster-samples-operator-697587599-g8q7v" (UID: "75d3929a-ac6f-4766-9496-9ae01f5ed769") : secret "samples-operator-tls" not found May 20 21:43:58.181171 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181101 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55be9913-a333-4eef-90a1-c2b3311cf9a6-service-ca-bundle\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.181540 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181293 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.181540 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181346 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/291a4427-81e3-47f4-b3cf-26ce3d5911b7-trusted-ca\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.181540 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181374 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0260fbc2-244d-4d26-80f6-55febb670445-serving-cert\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.181540 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4b6b\" (UniqueName: \"kubernetes.io/projected/0260fbc2-244d-4d26-80f6-55febb670445-kube-api-access-l4b6b\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.181761 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.181589 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46a4c312-92f4-4a59-8acd-96a2c204d00c-config\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.182078 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.182060 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/291a4427-81e3-47f4-b3cf-26ce3d5911b7-trusted-ca\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.183372 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.183353 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/291a4427-81e3-47f4-b3cf-26ce3d5911b7-serving-cert\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.183445 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.183389 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46a4c312-92f4-4a59-8acd-96a2c204d00c-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.186170 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.186154 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" May 20 21:43:58.190864 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.190840 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48"] May 20 21:43:58.191843 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.191823 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p"] May 20 21:43:58.193550 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.193531 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-kxsw4"] May 20 21:43:58.203213 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.203158 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzfg9\" (UniqueName: \"kubernetes.io/projected/46a4c312-92f4-4a59-8acd-96a2c204d00c-kube-api-access-qzfg9\") pod \"kube-storage-version-migrator-operator-565644f49-tlvqp\" (UID: \"46a4c312-92f4-4a59-8acd-96a2c204d00c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.203304 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.203252 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf7pm\" (UniqueName: \"kubernetes.io/projected/75d3929a-ac6f-4766-9496-9ae01f5ed769-kube-api-access-mf7pm\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.203304 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.203262 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw9w4\" (UniqueName: \"kubernetes.io/projected/cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad-kube-api-access-dw9w4\") pod \"volume-data-source-validator-5cf8c89d64-wcwl5\" (UID: \"cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" May 20 21:43:58.203379 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.203306 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw6dc\" (UniqueName: \"kubernetes.io/projected/291a4427-81e3-47f4-b3cf-26ce3d5911b7-kube-api-access-bw6dc\") pod \"console-operator-76dff48d59-44f4f\" (UID: \"291a4427-81e3-47f4-b3cf-26ce3d5911b7\") " pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.203835 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.203814 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkn2b\" (UniqueName: \"kubernetes.io/projected/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-kube-api-access-kkn2b\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.236210 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.236186 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6"] May 20 21:43:58.239911 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.239885 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" May 20 21:43:58.242779 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.242759 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-ncmnw\"" May 20 21:43:58.249527 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.249505 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6"] May 20 21:43:58.250406 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.250385 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58f97b448b-g9d6k"] May 20 21:43:58.252652 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.252633 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.255022 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255005 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" May 20 21:43:58.255215 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255181 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" May 20 21:43:58.255215 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255194 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" May 20 21:43:58.255321 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255198 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 20 21:43:58.255321 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255234 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" May 20 21:43:58.255470 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255458 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 20 21:43:58.255686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.255669 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7vfkk\"" May 20 21:43:58.266354 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.266336 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58f97b448b-g9d6k"] May 20 21:43:58.282103 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282082 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55be9913-a333-4eef-90a1-c2b3311cf9a6-service-ca-bundle\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.282214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282117 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.282214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282157 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0260fbc2-244d-4d26-80f6-55febb670445-serving-cert\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.282214 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282183 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l4b6b\" (UniqueName: \"kubernetes.io/projected/0260fbc2-244d-4d26-80f6-55febb670445-kube-api-access-l4b6b\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.282375 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282212 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.282375 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.282229 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 20 21:43:58.282375 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.282375 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282296 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnlbz\" (UniqueName: \"kubernetes.io/projected/150654bb-1c66-4f64-97dd-5a1cdd4d370f-kube-api-access-cnlbz\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.282375 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.282311 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert podName:09b71b40-b58b-4831-ac46-0f4c09f6f287 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:58.782292829 +0000 UTC m=+53.563726059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-9gf48" (UID: "09b71b40-b58b-4831-ac46-0f4c09f6f287") : secret "networking-console-plugin-cert" not found May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-stats-auth\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282417 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/55be9913-a333-4eef-90a1-c2b3311cf9a6-snapshots\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282443 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55be9913-a333-4eef-90a1-c2b3311cf9a6-serving-cert\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282476 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5zqc4\" (UniqueName: \"kubernetes.io/projected/55be9913-a333-4eef-90a1-c2b3311cf9a6-kube-api-access-5zqc4\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282510 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09b71b40-b58b-4831-ac46-0f4c09f6f287-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282547 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55be9913-a333-4eef-90a1-c2b3311cf9a6-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282609 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cswj2\" (UniqueName: \"kubernetes.io/projected/ee439e7b-e973-4ba2-97d7-69be8a05d10a-kube-api-access-cswj2\") pod \"network-check-source-7f44b6dd87-nnrj6\" (UID: \"ee439e7b-e973-4ba2-97d7-69be8a05d10a\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" May 20 21:43:58.282638 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282630 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55be9913-a333-4eef-90a1-c2b3311cf9a6-service-ca-bundle\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.283003 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282649 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0260fbc2-244d-4d26-80f6-55febb670445-config\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.283003 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282787 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55be9913-a333-4eef-90a1-c2b3311cf9a6-tmp\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.283108 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.283052 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/55be9913-a333-4eef-90a1-c2b3311cf9a6-snapshots\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.283166 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.283105 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55be9913-a333-4eef-90a1-c2b3311cf9a6-tmp\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.283166 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.282849 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-default-certificate\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.283258 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.283224 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0260fbc2-244d-4d26-80f6-55febb670445-config\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.283301 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.283251 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/09b71b40-b58b-4831-ac46-0f4c09f6f287-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.283536 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.283516 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/55be9913-a333-4eef-90a1-c2b3311cf9a6-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.284462 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.284442 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0260fbc2-244d-4d26-80f6-55febb670445-serving-cert\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.284800 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.284783 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/55be9913-a333-4eef-90a1-c2b3311cf9a6-serving-cert\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.292209 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.292190 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4b6b\" (UniqueName: \"kubernetes.io/projected/0260fbc2-244d-4d26-80f6-55febb670445-kube-api-access-l4b6b\") pod \"service-ca-operator-57b5b68b8-qq26p\" (UID: \"0260fbc2-244d-4d26-80f6-55febb670445\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.303701 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.303685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zqc4\" (UniqueName: \"kubernetes.io/projected/55be9913-a333-4eef-90a1-c2b3311cf9a6-kube-api-access-5zqc4\") pod \"insights-operator-78bdfb888c-kxsw4\" (UID: \"55be9913-a333-4eef-90a1-c2b3311cf9a6\") " pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.382407 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.382354 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" May 20 21:43:58.384341 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.384322 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cswj2\" (UniqueName: \"kubernetes.io/projected/ee439e7b-e973-4ba2-97d7-69be8a05d10a-kube-api-access-cswj2\") pod \"network-check-source-7f44b6dd87-nnrj6\" (UID: \"ee439e7b-e973-4ba2-97d7-69be8a05d10a\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" May 20 21:43:58.384419 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.384366 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-default-certificate\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.384419 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.384404 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.384530 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.384424 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.384530 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.384451 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cnlbz\" (UniqueName: \"kubernetes.io/projected/150654bb-1c66-4f64-97dd-5a1cdd4d370f-kube-api-access-cnlbz\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.384530 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.384484 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-stats-auth\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.384530 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.384508 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 20 21:43:58.384720 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.384592 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:58.884559648 +0000 UTC m=+53.665992887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : secret "router-metrics-certs-default" not found May 20 21:43:58.384720 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.384625 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:58.8846103 +0000 UTC m=+53.666043560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : configmap references non-existent config key: service-ca.crt May 20 21:43:58.386497 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.386470 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-default-certificate\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.386721 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.386704 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-stats-auth\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.387546 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.387527 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:43:58.393991 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.393957 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" May 20 21:43:58.396709 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.396685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cswj2\" (UniqueName: \"kubernetes.io/projected/ee439e7b-e973-4ba2-97d7-69be8a05d10a-kube-api-access-cswj2\") pod \"network-check-source-7f44b6dd87-nnrj6\" (UID: \"ee439e7b-e973-4ba2-97d7-69be8a05d10a\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" May 20 21:43:58.396814 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.396786 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnlbz\" (UniqueName: \"kubernetes.io/projected/150654bb-1c66-4f64-97dd-5a1cdd4d370f-kube-api-access-cnlbz\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.498151 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.498123 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" May 20 21:43:58.504386 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.504352 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" May 20 21:43:58.523360 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.523330 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5"] May 20 21:43:58.527963 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:58.527927 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad1b37a_2384_40f9_aba0_8d8f1f9ee0ad.slice/crio-7285a2fc58e292776c520cedc7cbca7fa09cb56c5f31ee83706907001f783a40 WatchSource:0}: Error finding container 7285a2fc58e292776c520cedc7cbca7fa09cb56c5f31ee83706907001f783a40: Status 404 returned error can't find the container with id 7285a2fc58e292776c520cedc7cbca7fa09cb56c5f31ee83706907001f783a40 May 20 21:43:58.547148 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.547123 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-44f4f"] May 20 21:43:58.548884 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.548858 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" May 20 21:43:58.550244 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:58.550202 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291a4427_81e3_47f4_b3cf_26ce3d5911b7.slice/crio-b4512d80b0380f2568c73e8d4fbb3933d834caec8e4985465c088472feac96d6 WatchSource:0}: Error finding container b4512d80b0380f2568c73e8d4fbb3933d834caec8e4985465c088472feac96d6: Status 404 returned error can't find the container with id b4512d80b0380f2568c73e8d4fbb3933d834caec8e4985465c088472feac96d6 May 20 21:43:58.563962 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.563917 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp"] May 20 21:43:58.576344 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:58.576306 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a4c312_92f4_4a59_8acd_96a2c204d00c.slice/crio-18888a266a35fb265276737e2893ed9babd8730a82e48e1346e1ac4e92d03716 WatchSource:0}: Error finding container 18888a266a35fb265276737e2893ed9babd8730a82e48e1346e1ac4e92d03716: Status 404 returned error can't find the container with id 18888a266a35fb265276737e2893ed9babd8730a82e48e1346e1ac4e92d03716 May 20 21:43:58.647123 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.647073 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p"] May 20 21:43:58.649438 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:58.649409 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0260fbc2_244d_4d26_80f6_55febb670445.slice/crio-8d1100472eb029c462a6763d5c1f0aa1ca266b6628669916c8c2910af1bfe6c1 WatchSource:0}: Error finding container 8d1100472eb029c462a6763d5c1f0aa1ca266b6628669916c8c2910af1bfe6c1: Status 404 returned error can't find the container with id 8d1100472eb029c462a6763d5c1f0aa1ca266b6628669916c8c2910af1bfe6c1 May 20 21:43:58.668088 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.668060 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-kxsw4"] May 20 21:43:58.671100 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:58.671069 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55be9913_a333_4eef_90a1_c2b3311cf9a6.slice/crio-7caa14e2577c372837abb8ceb057d6ee73fdd15807120beebac35026afda2fe1 WatchSource:0}: Error finding container 7caa14e2577c372837abb8ceb057d6ee73fdd15807120beebac35026afda2fe1: Status 404 returned error can't find the container with id 7caa14e2577c372837abb8ceb057d6ee73fdd15807120beebac35026afda2fe1 May 20 21:43:58.688235 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.688213 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:58.688329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.688301 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:58.688402 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.688379 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 20 21:43:58.688454 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.688386 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 20 21:43:58.688454 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.688447 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls podName:7eb15ff5-c1cf-40e2-a612-7e5494cc395e nodeName:}" failed. No retries permitted until 2026-05-20 21:43:59.688430382 +0000 UTC m=+54.469863611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-72ztf" (UID: "7eb15ff5-c1cf-40e2-a612-7e5494cc395e") : secret "cluster-monitoring-operator-tls" not found May 20 21:43:58.688560 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.688469 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls podName:75d3929a-ac6f-4766-9496-9ae01f5ed769 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:59.688456949 +0000 UTC m=+54.469890181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls") pod "cluster-samples-operator-697587599-g8q7v" (UID: "75d3929a-ac6f-4766-9496-9ae01f5ed769") : secret "samples-operator-tls" not found May 20 21:43:58.691335 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.691314 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6"] May 20 21:43:58.694394 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:43:58.694371 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee439e7b_e973_4ba2_97d7_69be8a05d10a.slice/crio-f46cacd2f34580a2477817d6794b174d4b9e46ba0778cbd4fdad9e6c40653db0 WatchSource:0}: Error finding container f46cacd2f34580a2477817d6794b174d4b9e46ba0778cbd4fdad9e6c40653db0: Status 404 returned error can't find the container with id f46cacd2f34580a2477817d6794b174d4b9e46ba0778cbd4fdad9e6c40653db0 May 20 21:43:58.789679 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.789652 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:58.789803 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.789786 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 20 21:43:58.789852 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.789843 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert podName:09b71b40-b58b-4831-ac46-0f4c09f6f287 nodeName:}" failed. No retries permitted until 2026-05-20 21:43:59.789827914 +0000 UTC m=+54.571261145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-9gf48" (UID: "09b71b40-b58b-4831-ac46-0f4c09f6f287") : secret "networking-console-plugin-cert" not found May 20 21:43:58.891007 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.890979 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.891116 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:58.891017 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:58.891116 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.891104 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 20 21:43:58.891195 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.891151 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:59.891137951 +0000 UTC m=+54.672571181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : configmap references non-existent config key: service-ca.crt May 20 21:43:58.891195 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:58.891166 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:43:59.891160314 +0000 UTC m=+54.672593543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : secret "router-metrics-certs-default" not found May 20 21:43:59.059819 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.059789 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" event={"ID":"55be9913-a333-4eef-90a1-c2b3311cf9a6","Type":"ContainerStarted","Data":"7caa14e2577c372837abb8ceb057d6ee73fdd15807120beebac35026afda2fe1"} May 20 21:43:59.060660 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.060639 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" event={"ID":"ee439e7b-e973-4ba2-97d7-69be8a05d10a","Type":"ContainerStarted","Data":"f46cacd2f34580a2477817d6794b174d4b9e46ba0778cbd4fdad9e6c40653db0"} May 20 21:43:59.061474 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.061453 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" event={"ID":"46a4c312-92f4-4a59-8acd-96a2c204d00c","Type":"ContainerStarted","Data":"18888a266a35fb265276737e2893ed9babd8730a82e48e1346e1ac4e92d03716"} May 20 21:43:59.062317 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.062300 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" event={"ID":"291a4427-81e3-47f4-b3cf-26ce3d5911b7","Type":"ContainerStarted","Data":"b4512d80b0380f2568c73e8d4fbb3933d834caec8e4985465c088472feac96d6"} May 20 21:43:59.063227 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.063209 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" event={"ID":"0260fbc2-244d-4d26-80f6-55febb670445","Type":"ContainerStarted","Data":"8d1100472eb029c462a6763d5c1f0aa1ca266b6628669916c8c2910af1bfe6c1"} May 20 21:43:59.063987 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.063970 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" event={"ID":"cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad","Type":"ContainerStarted","Data":"7285a2fc58e292776c520cedc7cbca7fa09cb56c5f31ee83706907001f783a40"} May 20 21:43:59.700125 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.700070 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:43:59.700313 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.700190 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:43:59.700429 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.700396 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 20 21:43:59.700483 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.700458 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls podName:75d3929a-ac6f-4766-9496-9ae01f5ed769 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:01.700439185 +0000 UTC m=+56.481872416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls") pod "cluster-samples-operator-697587599-g8q7v" (UID: "75d3929a-ac6f-4766-9496-9ae01f5ed769") : secret "samples-operator-tls" not found May 20 21:43:59.700910 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.700890 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 20 21:43:59.701006 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.700940 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls podName:7eb15ff5-c1cf-40e2-a612-7e5494cc395e nodeName:}" failed. No retries permitted until 2026-05-20 21:44:01.700925952 +0000 UTC m=+56.482359182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-72ztf" (UID: "7eb15ff5-c1cf-40e2-a612-7e5494cc395e") : secret "cluster-monitoring-operator-tls" not found May 20 21:43:59.801201 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.801163 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:43:59.801388 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.801352 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 20 21:43:59.801454 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.801413 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert podName:09b71b40-b58b-4831-ac46-0f4c09f6f287 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:01.801393677 +0000 UTC m=+56.582826929 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-9gf48" (UID: "09b71b40-b58b-4831-ac46-0f4c09f6f287") : secret "networking-console-plugin-cert" not found May 20 21:43:59.902094 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.902053 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:59.902445 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:43:59.902117 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:43:59.902445 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.902354 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:01.902336392 +0000 UTC m=+56.683769623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : configmap references non-existent config key: service-ca.crt May 20 21:43:59.902938 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.902760 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 20 21:43:59.902938 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:43:59.902812 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:01.902796204 +0000 UTC m=+56.684229442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : secret "router-metrics-certs-default" not found May 20 21:44:01.721116 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:01.721082 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:44:01.721523 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:01.721176 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:44:01.721523 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.721249 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 20 21:44:01.721523 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.721321 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls podName:7eb15ff5-c1cf-40e2-a612-7e5494cc395e nodeName:}" failed. No retries permitted until 2026-05-20 21:44:05.721304833 +0000 UTC m=+60.502738064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-72ztf" (UID: "7eb15ff5-c1cf-40e2-a612-7e5494cc395e") : secret "cluster-monitoring-operator-tls" not found May 20 21:44:01.721523 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.721328 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 20 21:44:01.721523 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.721399 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls podName:75d3929a-ac6f-4766-9496-9ae01f5ed769 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:05.721380022 +0000 UTC m=+60.502813254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls") pod "cluster-samples-operator-697587599-g8q7v" (UID: "75d3929a-ac6f-4766-9496-9ae01f5ed769") : secret "samples-operator-tls" not found May 20 21:44:01.821915 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:01.821872 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:44:01.822084 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.822052 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 20 21:44:01.822152 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.822131 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert podName:09b71b40-b58b-4831-ac46-0f4c09f6f287 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:05.822109249 +0000 UTC m=+60.603542495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-9gf48" (UID: "09b71b40-b58b-4831-ac46-0f4c09f6f287") : secret "networking-console-plugin-cert" not found May 20 21:44:01.922796 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:01.922757 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:01.922990 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:01.922819 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:01.922990 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.922888 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 20 21:44:01.922990 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.922961 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:05.92294266 +0000 UTC m=+60.704375890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : secret "router-metrics-certs-default" not found May 20 21:44:01.922990 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:01.922981 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:05.922971185 +0000 UTC m=+60.704404415 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : configmap references non-existent config key: service-ca.crt May 20 21:44:05.088294 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.088256 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" event={"ID":"ee439e7b-e973-4ba2-97d7-69be8a05d10a","Type":"ContainerStarted","Data":"59429364b78f14c77f6a66b47fca208d754ec2aa56a5bfaeec96498519c9cd3a"} May 20 21:44:05.089880 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.089846 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" event={"ID":"46a4c312-92f4-4a59-8acd-96a2c204d00c","Type":"ContainerStarted","Data":"a06301ac596af6d5a6e108b3288cf557afd06f1f746db31f92a6296f6410810e"} May 20 21:44:05.091480 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.091460 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/0.log" May 20 21:44:05.091594 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.091496 2565 generic.go:358] "Generic (PLEG): container finished" podID="291a4427-81e3-47f4-b3cf-26ce3d5911b7" containerID="6483cab9471f6d8d228289429241462eb6a832df17d30261837804d484c5086a" exitCode=255 May 20 21:44:05.091670 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.091591 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" event={"ID":"291a4427-81e3-47f4-b3cf-26ce3d5911b7","Type":"ContainerDied","Data":"6483cab9471f6d8d228289429241462eb6a832df17d30261837804d484c5086a"} May 20 21:44:05.091846 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.091830 2565 scope.go:117] "RemoveContainer" containerID="6483cab9471f6d8d228289429241462eb6a832df17d30261837804d484c5086a" May 20 21:44:05.093065 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.093045 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" event={"ID":"0260fbc2-244d-4d26-80f6-55febb670445","Type":"ContainerStarted","Data":"65d40b3391a31a4b3201b30c8ea177196fd697dee4190e284dd5857a877b5587"} May 20 21:44:05.094686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.094664 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" event={"ID":"cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad","Type":"ContainerStarted","Data":"b374061c059f0d3c6a74330a9996b1e7e683816bc132d87368b3bf7fa999ae51"} May 20 21:44:05.096460 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.096080 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" event={"ID":"55be9913-a333-4eef-90a1-c2b3311cf9a6","Type":"ContainerStarted","Data":"e365942d84c426f893afdc6fe486663c8245aee185ad3d024c56344d515c759d"} May 20 21:44:05.114625 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.114557 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-nnrj6" podStartSLOduration=1.5621538799999999 podStartE2EDuration="7.114541027s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:43:58.696033503 +0000 UTC m=+53.477466736" lastFinishedPulling="2026-05-20 21:44:04.248420641 +0000 UTC m=+59.029853883" observedRunningTime="2026-05-20 21:44:05.113256845 +0000 UTC m=+59.894690099" watchObservedRunningTime="2026-05-20 21:44:05.114541027 +0000 UTC m=+59.895974279" May 20 21:44:05.138498 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.138188 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" podStartSLOduration=1.645384726 podStartE2EDuration="7.138171214s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:43:58.651338691 +0000 UTC m=+53.432771924" lastFinishedPulling="2026-05-20 21:44:04.144125183 +0000 UTC m=+58.925558412" observedRunningTime="2026-05-20 21:44:05.136294333 +0000 UTC m=+59.917727585" watchObservedRunningTime="2026-05-20 21:44:05.138171214 +0000 UTC m=+59.919604465" May 20 21:44:05.159856 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.159359 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-wcwl5" podStartSLOduration=1.565860357 podStartE2EDuration="7.159342767s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:43:58.530931858 +0000 UTC m=+53.312365087" lastFinishedPulling="2026-05-20 21:44:04.124414267 +0000 UTC m=+58.905847497" observedRunningTime="2026-05-20 21:44:05.157457543 +0000 UTC m=+59.938890793" watchObservedRunningTime="2026-05-20 21:44:05.159342767 +0000 UTC m=+59.940776019" May 20 21:44:05.197146 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.197088 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" podStartSLOduration=1.721554118 podStartE2EDuration="7.197069456s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:43:58.674032447 +0000 UTC m=+53.455465678" lastFinishedPulling="2026-05-20 21:44:04.149547786 +0000 UTC m=+58.930981016" observedRunningTime="2026-05-20 21:44:05.19580916 +0000 UTC m=+59.977242412" watchObservedRunningTime="2026-05-20 21:44:05.197069456 +0000 UTC m=+59.978502709" May 20 21:44:05.240819 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.240776 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" podStartSLOduration=1.674924673 podStartE2EDuration="7.240761809s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:43:58.578461181 +0000 UTC m=+53.359894410" lastFinishedPulling="2026-05-20 21:44:04.144298313 +0000 UTC m=+58.925731546" observedRunningTime="2026-05-20 21:44:05.239374806 +0000 UTC m=+60.020808050" watchObservedRunningTime="2026-05-20 21:44:05.240761809 +0000 UTC m=+60.022195058" May 20 21:44:05.766187 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.766148 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:44:05.766366 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.766294 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found May 20 21:44:05.766366 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.766311 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:44:05.766366 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.766361 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls podName:75d3929a-ac6f-4766-9496-9ae01f5ed769 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:13.766340988 +0000 UTC m=+68.547774223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls") pod "cluster-samples-operator-697587599-g8q7v" (UID: "75d3929a-ac6f-4766-9496-9ae01f5ed769") : secret "samples-operator-tls" not found May 20 21:44:05.766514 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.766466 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 20 21:44:05.766561 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.766532 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls podName:7eb15ff5-c1cf-40e2-a612-7e5494cc395e nodeName:}" failed. No retries permitted until 2026-05-20 21:44:13.766513502 +0000 UTC m=+68.547946733 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-72ztf" (UID: "7eb15ff5-c1cf-40e2-a612-7e5494cc395e") : secret "cluster-monitoring-operator-tls" not found May 20 21:44:05.810687 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.810654 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw"] May 20 21:44:05.813698 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.813671 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" May 20 21:44:05.816605 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.816584 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-bq9hp\"" May 20 21:44:05.816726 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.816703 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" May 20 21:44:05.817112 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.817094 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" May 20 21:44:05.839732 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.839705 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw"] May 20 21:44:05.867145 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.867109 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:44:05.867288 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.867208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r75pk\" (UniqueName: \"kubernetes.io/projected/a20fd981-b049-45dc-bf1f-1180f4205203-kube-api-access-r75pk\") pod \"migrator-b985f5d89-qrwmw\" (UID: \"a20fd981-b049-45dc-bf1f-1180f4205203\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" May 20 21:44:05.867288 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.867274 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 20 21:44:05.867368 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.867347 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert podName:09b71b40-b58b-4831-ac46-0f4c09f6f287 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:13.867327118 +0000 UTC m=+68.648760353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-9gf48" (UID: "09b71b40-b58b-4831-ac46-0f4c09f6f287") : secret "networking-console-plugin-cert" not found May 20 21:44:05.967808 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.967776 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:05.967808 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.967817 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:05.968000 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.967925 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 20 21:44:05.968000 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.967936 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:13.967918168 +0000 UTC m=+68.749351416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : configmap references non-existent config key: service-ca.crt May 20 21:44:05.968091 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.968023 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r75pk\" (UniqueName: \"kubernetes.io/projected/a20fd981-b049-45dc-bf1f-1180f4205203-kube-api-access-r75pk\") pod \"migrator-b985f5d89-qrwmw\" (UID: \"a20fd981-b049-45dc-bf1f-1180f4205203\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" May 20 21:44:05.968091 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:05.968054 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:13.968028553 +0000 UTC m=+68.749461790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : secret "router-metrics-certs-default" not found May 20 21:44:05.977690 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.977664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r75pk\" (UniqueName: \"kubernetes.io/projected/a20fd981-b049-45dc-bf1f-1180f4205203-kube-api-access-r75pk\") pod \"migrator-b985f5d89-qrwmw\" (UID: \"a20fd981-b049-45dc-bf1f-1180f4205203\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" May 20 21:44:05.981094 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:05.981068 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6m8q" May 20 21:44:06.100563 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.100538 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:44:06.101048 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.101010 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/0.log" May 20 21:44:06.101114 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.101078 2565 generic.go:358] "Generic (PLEG): container finished" podID="291a4427-81e3-47f4-b3cf-26ce3d5911b7" containerID="adda8d270de60924284486a40596c52f8d4f6877a1b153baa80f6a95c99c4d39" exitCode=255 May 20 21:44:06.101236 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.101169 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" event={"ID":"291a4427-81e3-47f4-b3cf-26ce3d5911b7","Type":"ContainerDied","Data":"adda8d270de60924284486a40596c52f8d4f6877a1b153baa80f6a95c99c4d39"} May 20 21:44:06.101236 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.101219 2565 scope.go:117] "RemoveContainer" containerID="6483cab9471f6d8d228289429241462eb6a832df17d30261837804d484c5086a" May 20 21:44:06.101488 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.101466 2565 scope.go:117] "RemoveContainer" containerID="adda8d270de60924284486a40596c52f8d4f6877a1b153baa80f6a95c99c4d39" May 20 21:44:06.101728 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:06.101704 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-44f4f_openshift-console-operator(291a4427-81e3-47f4-b3cf-26ce3d5911b7)\"" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" podUID="291a4427-81e3-47f4-b3cf-26ce3d5911b7" May 20 21:44:06.123479 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.123457 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" May 20 21:44:06.270253 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.270207 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw"] May 20 21:44:06.273087 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:06.273061 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda20fd981_b049_45dc_bf1f_1180f4205203.slice/crio-c43c4db6f1df638256d93f1823a9ae9bb9d0f67225f215f12f0a9088fa476823 WatchSource:0}: Error finding container c43c4db6f1df638256d93f1823a9ae9bb9d0f67225f215f12f0a9088fa476823: Status 404 returned error can't find the container with id c43c4db6f1df638256d93f1823a9ae9bb9d0f67225f215f12f0a9088fa476823 May 20 21:44:06.522425 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:06.522354 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-47h4c_c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2/dns-node-resolver/0.log" May 20 21:44:07.105274 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.105238 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" event={"ID":"a20fd981-b049-45dc-bf1f-1180f4205203","Type":"ContainerStarted","Data":"c43c4db6f1df638256d93f1823a9ae9bb9d0f67225f215f12f0a9088fa476823"} May 20 21:44:07.106340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.106323 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:44:07.106687 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.106673 2565 scope.go:117] "RemoveContainer" containerID="adda8d270de60924284486a40596c52f8d4f6877a1b153baa80f6a95c99c4d39" May 20 21:44:07.106852 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:07.106836 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-44f4f_openshift-console-operator(291a4427-81e3-47f4-b3cf-26ce3d5911b7)\"" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" podUID="291a4427-81e3-47f4-b3cf-26ce3d5911b7" May 20 21:44:07.725828 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.725752 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-546st"] May 20 21:44:07.727915 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.727899 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.730762 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.730736 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" May 20 21:44:07.730890 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.730807 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" May 20 21:44:07.730890 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.730835 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-784df\"" May 20 21:44:07.731075 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.731061 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" May 20 21:44:07.731114 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.731064 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" May 20 21:44:07.738264 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.738243 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-546st"] May 20 21:44:07.786124 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.786091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8c55a443-79d6-4b86-b8bd-99db703e4bd6-signing-cabundle\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.786290 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.786163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8c55a443-79d6-4b86-b8bd-99db703e4bd6-signing-key\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.786290 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.786204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5dfq\" (UniqueName: \"kubernetes.io/projected/8c55a443-79d6-4b86-b8bd-99db703e4bd6-kube-api-access-v5dfq\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.887611 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.887512 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8c55a443-79d6-4b86-b8bd-99db703e4bd6-signing-cabundle\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.887766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.887659 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8c55a443-79d6-4b86-b8bd-99db703e4bd6-signing-key\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.887766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.887731 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v5dfq\" (UniqueName: \"kubernetes.io/projected/8c55a443-79d6-4b86-b8bd-99db703e4bd6-kube-api-access-v5dfq\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.891174 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.888938 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8c55a443-79d6-4b86-b8bd-99db703e4bd6-signing-cabundle\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.891174 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.890938 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8c55a443-79d6-4b86-b8bd-99db703e4bd6-signing-key\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.895738 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.895714 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5dfq\" (UniqueName: \"kubernetes.io/projected/8c55a443-79d6-4b86-b8bd-99db703e4bd6-kube-api-access-v5dfq\") pod \"service-ca-67d667c7d-546st\" (UID: \"8c55a443-79d6-4b86-b8bd-99db703e4bd6\") " pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:07.922926 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:07.922899 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-knkhq_64b82855-f1bb-4887-8207-26154822ff4e/node-ca/0.log" May 20 21:44:08.036927 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.036837 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-546st" May 20 21:44:08.110274 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.110234 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" event={"ID":"a20fd981-b049-45dc-bf1f-1180f4205203","Type":"ContainerStarted","Data":"ea3d915ac3ff78c0ca74f75f98f28dc05969cb7cf99f87022c8f61d5ccb7545a"} May 20 21:44:08.110274 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.110273 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" event={"ID":"a20fd981-b049-45dc-bf1f-1180f4205203","Type":"ContainerStarted","Data":"2c8dd9d1be4de78e5a763b48c00c965b624d6250984170f1995eaa2528fa7018"} May 20 21:44:08.131309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.131254 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-qrwmw" podStartSLOduration=1.949121734 podStartE2EDuration="3.131234399s" podCreationTimestamp="2026-05-20 21:44:05 +0000 UTC" firstStartedPulling="2026-05-20 21:44:06.275486051 +0000 UTC m=+61.056919284" lastFinishedPulling="2026-05-20 21:44:07.457598718 +0000 UTC m=+62.239031949" observedRunningTime="2026-05-20 21:44:08.130463636 +0000 UTC m=+62.911896887" watchObservedRunningTime="2026-05-20 21:44:08.131234399 +0000 UTC m=+62.912667659" May 20 21:44:08.157208 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.157153 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-546st"] May 20 21:44:08.159189 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:08.159159 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c55a443_79d6_4b86_b8bd_99db703e4bd6.slice/crio-0402e5442d6380fbe00f9f3cd1e17e46a1fbda15ed017bf7a51ed22f614ae9ff WatchSource:0}: Error finding container 0402e5442d6380fbe00f9f3cd1e17e46a1fbda15ed017bf7a51ed22f614ae9ff: Status 404 returned error can't find the container with id 0402e5442d6380fbe00f9f3cd1e17e46a1fbda15ed017bf7a51ed22f614ae9ff May 20 21:44:08.388106 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.388073 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:44:08.388106 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.388115 2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:44:08.388501 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:08.388488 2565 scope.go:117] "RemoveContainer" containerID="adda8d270de60924284486a40596c52f8d4f6877a1b153baa80f6a95c99c4d39" May 20 21:44:08.388695 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:08.388673 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-44f4f_openshift-console-operator(291a4427-81e3-47f4-b3cf-26ce3d5911b7)\"" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" podUID="291a4427-81e3-47f4-b3cf-26ce3d5911b7" May 20 21:44:09.115073 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:09.115040 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-546st" event={"ID":"8c55a443-79d6-4b86-b8bd-99db703e4bd6","Type":"ContainerStarted","Data":"6cf3102a6af4c85155bc3b8ccdd8accfd707595b5969fbe374fba4b2ce0242a3"} May 20 21:44:09.115073 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:09.115074 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-546st" event={"ID":"8c55a443-79d6-4b86-b8bd-99db703e4bd6","Type":"ContainerStarted","Data":"0402e5442d6380fbe00f9f3cd1e17e46a1fbda15ed017bf7a51ed22f614ae9ff"} May 20 21:44:09.134881 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:09.134837 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-546st" podStartSLOduration=2.134824903 podStartE2EDuration="2.134824903s" podCreationTimestamp="2026-05-20 21:44:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:44:09.134086849 +0000 UTC m=+63.915520109" watchObservedRunningTime="2026-05-20 21:44:09.134824903 +0000 UTC m=+63.916258156" May 20 21:44:11.520298 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.520262 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:44:11.520678 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.520311 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:44:11.522986 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.522956 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxkjz\" (UniqueName: \"kubernetes.io/projected/ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5-kube-api-access-wxkjz\") pod \"network-check-target-xj72f\" (UID: \"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5\") " pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:44:11.523300 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.523283 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 20 21:44:11.530891 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:11.530873 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 20 21:44:11.530980 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:11.530936 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs podName:a5df8c31-3ae1-4ae3-9a87-3da35016fd6f nodeName:}" failed. No retries permitted until 2026-05-20 21:45:15.530917405 +0000 UTC m=+130.312350647 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs") pod "network-metrics-daemon-2slgv" (UID: "a5df8c31-3ae1-4ae3-9a87-3da35016fd6f") : secret "metrics-daemon-secret" not found May 20 21:44:11.755180 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.755151 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sgd7q\"" May 20 21:44:11.763454 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.763432 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:44:11.888518 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:11.888485 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xj72f"] May 20 21:44:11.893675 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:11.893643 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca8a0d4c_2860_4861_bdd0_dc35eb3fcbd5.slice/crio-248be0879a2cb191baee48499cd59669fa81a10c4c9d6a4653f898ee0d5c469e WatchSource:0}: Error finding container 248be0879a2cb191baee48499cd59669fa81a10c4c9d6a4653f898ee0d5c469e: Status 404 returned error can't find the container with id 248be0879a2cb191baee48499cd59669fa81a10c4c9d6a4653f898ee0d5c469e May 20 21:44:12.126979 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.126948 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xj72f" event={"ID":"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5","Type":"ContainerStarted","Data":"6477be79682b81502d0df0c0bae00a97e7544dca7ff28cdfcb2ad12dd5e936d5"} May 20 21:44:12.126979 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.126985 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xj72f" event={"ID":"ca8a0d4c-2860-4861-bdd0-dc35eb3fcbd5","Type":"ContainerStarted","Data":"248be0879a2cb191baee48499cd59669fa81a10c4c9d6a4653f898ee0d5c469e"} May 20 21:44:12.127188 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.127090 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:44:12.147115 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.147064 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xj72f" podStartSLOduration=67.147048431 podStartE2EDuration="1m7.147048431s" podCreationTimestamp="2026-05-20 21:43:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:44:12.145661866 +0000 UTC m=+66.927095140" watchObservedRunningTime="2026-05-20 21:44:12.147048431 +0000 UTC m=+66.928481682" May 20 21:44:12.529586 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.529496 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:44:12.529586 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.529558 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:44:12.530002 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.529659 2565 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 20 21:44:12.530002 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.529679 2565 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d49b96d4b-jdts2: secret "image-registry-tls" not found May 20 21:44:12.530002 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.529721 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 20 21:44:12.530002 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.529732 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls podName:a7dbaa90-54fe-4da4-904c-e6f6c35d9e89 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:44.529716671 +0000 UTC m=+99.311149902 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls") pod "image-registry-d49b96d4b-jdts2" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89") : secret "image-registry-tls" not found May 20 21:44:12.530002 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.529764 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls podName:d1eddfa4-fc63-448b-b6f9-2e321d8b02f2 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:44.529752919 +0000 UTC m=+99.311186152 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls") pod "dns-default-6xhk7" (UID: "d1eddfa4-fc63-448b-b6f9-2e321d8b02f2") : secret "dns-default-metrics-tls" not found May 20 21:44:12.630724 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:12.630694 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:44:12.630867 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.630823 2565 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 20 21:44:12.630906 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:12.630878 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert podName:7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b nodeName:}" failed. No retries permitted until 2026-05-20 21:44:44.63086467 +0000 UTC m=+99.412297900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert") pod "ingress-canary-6hfxx" (UID: "7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b") : secret "canary-serving-cert" not found May 20 21:44:13.841252 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:13.841206 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:44:13.841747 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:13.841315 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:44:13.841747 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:13.841368 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found May 20 21:44:13.841747 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:13.841456 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls podName:7eb15ff5-c1cf-40e2-a612-7e5494cc395e nodeName:}" failed. No retries permitted until 2026-05-20 21:44:29.841435564 +0000 UTC m=+84.622868808 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-72ztf" (UID: "7eb15ff5-c1cf-40e2-a612-7e5494cc395e") : secret "cluster-monitoring-operator-tls" not found May 20 21:44:13.843694 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:13.843676 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/75d3929a-ac6f-4766-9496-9ae01f5ed769-samples-operator-tls\") pod \"cluster-samples-operator-697587599-g8q7v\" (UID: \"75d3929a-ac6f-4766-9496-9ae01f5ed769\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:44:13.942073 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:13.942030 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:44:13.942262 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:13.942174 2565 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found May 20 21:44:13.942262 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:13.942248 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert podName:09b71b40-b58b-4831-ac46-0f4c09f6f287 nodeName:}" failed. No retries permitted until 2026-05-20 21:44:29.942222003 +0000 UTC m=+84.723655238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-9gf48" (UID: "09b71b40-b58b-4831-ac46-0f4c09f6f287") : secret "networking-console-plugin-cert" not found May 20 21:44:14.003628 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:14.003602 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7gm98\"" May 20 21:44:14.011823 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:14.011805 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" May 20 21:44:14.042786 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:14.042746 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:14.042786 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:14.042788 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:14.042988 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:14.042918 2565 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found May 20 21:44:14.042988 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:14.042956 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:30.04294329 +0000 UTC m=+84.824376520 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : configmap references non-existent config key: service-ca.crt May 20 21:44:14.043103 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:44:14.042991 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs podName:150654bb-1c66-4f64-97dd-5a1cdd4d370f nodeName:}" failed. No retries permitted until 2026-05-20 21:44:30.042973849 +0000 UTC m=+84.824407096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs") pod "router-default-58f97b448b-g9d6k" (UID: "150654bb-1c66-4f64-97dd-5a1cdd4d370f") : secret "router-metrics-certs-default" not found May 20 21:44:14.161836 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:14.161759 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v"] May 20 21:44:15.135659 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:15.135623 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" event={"ID":"75d3929a-ac6f-4766-9496-9ae01f5ed769","Type":"ContainerStarted","Data":"bcd1d43417da50813b8b8d61c68e054ec47c7f46362b59fce2010e8845bbc7a1"} May 20 21:44:16.140337 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:16.140298 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" event={"ID":"75d3929a-ac6f-4766-9496-9ae01f5ed769","Type":"ContainerStarted","Data":"1c1ba03d5d4d43fcc58707b022834a5a2f742e686bcbe27eff734a5010e82787"} May 20 21:44:16.140337 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:16.140342 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" event={"ID":"75d3929a-ac6f-4766-9496-9ae01f5ed769","Type":"ContainerStarted","Data":"0c8ce652771a7187a783d707be546bac6d7ad2349eb94e1c1f8caa542041a312"} May 20 21:44:21.742691 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:21.742662 2565 scope.go:117] "RemoveContainer" containerID="adda8d270de60924284486a40596c52f8d4f6877a1b153baa80f6a95c99c4d39" May 20 21:44:22.160817 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:22.160788 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:44:22.160995 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:22.160878 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" event={"ID":"291a4427-81e3-47f4-b3cf-26ce3d5911b7","Type":"ContainerStarted","Data":"a0ada63b40865ebcfcfb4e30bc5e3045c52ceb4162eb0fb623a9a1125d86cd57"} May 20 21:44:22.161260 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:22.161236 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:44:22.185012 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:22.184967 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-g8q7v" podStartSLOduration=22.627363368 podStartE2EDuration="24.184955917s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:44:14.208959268 +0000 UTC m=+68.990392497" lastFinishedPulling="2026-05-20 21:44:15.766551817 +0000 UTC m=+70.547985046" observedRunningTime="2026-05-20 21:44:16.18037563 +0000 UTC m=+70.961808881" watchObservedRunningTime="2026-05-20 21:44:22.184955917 +0000 UTC m=+76.966389169" May 20 21:44:22.185229 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:22.185208 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" podStartSLOduration=18.595450435 podStartE2EDuration="24.18520319s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:43:58.554227989 +0000 UTC m=+53.335661218" lastFinishedPulling="2026-05-20 21:44:04.14398074 +0000 UTC m=+58.925413973" observedRunningTime="2026-05-20 21:44:22.184657217 +0000 UTC m=+76.966090489" watchObservedRunningTime="2026-05-20 21:44:22.18520319 +0000 UTC m=+76.966636442" May 20 21:44:22.777923 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:22.777876 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-44f4f" May 20 21:44:28.860297 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.860182 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-2nbqz"] May 20 21:44:28.862946 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.862929 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:28.868467 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.868444 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 20 21:44:28.868600 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.868469 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 20 21:44:28.874775 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.874759 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-95v4t\"" May 20 21:44:28.924367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.924343 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-2nbqz"] May 20 21:44:28.964247 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.964218 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9v4xs"] May 20 21:44:28.966941 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.966922 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:28.967202 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.967178 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4hrz\" (UniqueName: \"kubernetes.io/projected/a584980a-67ce-4416-b2d0-180c64eafacd-kube-api-access-q4hrz\") pod \"downloads-57bd96b888-2nbqz\" (UID: \"a584980a-67ce-4416-b2d0-180c64eafacd\") " pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:28.970297 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.970262 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 20 21:44:28.970408 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.970374 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6ms9h\"" May 20 21:44:28.970779 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.970761 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 20 21:44:28.988437 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:28.988417 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9v4xs"] May 20 21:44:29.068088 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.068056 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a88b42d3-8f3d-4847-861e-f33c7c04274d-crio-socket\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.068254 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.068106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a88b42d3-8f3d-4847-861e-f33c7c04274d-data-volume\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.068254 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.068182 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a88b42d3-8f3d-4847-861e-f33c7c04274d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.068254 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.068227 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4pnk\" (UniqueName: \"kubernetes.io/projected/a88b42d3-8f3d-4847-861e-f33c7c04274d-kube-api-access-n4pnk\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.068368 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.068273 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q4hrz\" (UniqueName: \"kubernetes.io/projected/a584980a-67ce-4416-b2d0-180c64eafacd-kube-api-access-q4hrz\") pod \"downloads-57bd96b888-2nbqz\" (UID: \"a584980a-67ce-4416-b2d0-180c64eafacd\") " pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:29.068368 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.068354 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a88b42d3-8f3d-4847-861e-f33c7c04274d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.087711 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.087685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4hrz\" (UniqueName: \"kubernetes.io/projected/a584980a-67ce-4416-b2d0-180c64eafacd-kube-api-access-q4hrz\") pod \"downloads-57bd96b888-2nbqz\" (UID: \"a584980a-67ce-4416-b2d0-180c64eafacd\") " pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:29.168828 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.168746 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a88b42d3-8f3d-4847-861e-f33c7c04274d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.168828 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.168792 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a88b42d3-8f3d-4847-861e-f33c7c04274d-crio-socket\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.169002 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.168857 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a88b42d3-8f3d-4847-861e-f33c7c04274d-crio-socket\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.169002 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.168913 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a88b42d3-8f3d-4847-861e-f33c7c04274d-data-volume\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.169002 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.168946 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a88b42d3-8f3d-4847-861e-f33c7c04274d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.169002 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.168973 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n4pnk\" (UniqueName: \"kubernetes.io/projected/a88b42d3-8f3d-4847-861e-f33c7c04274d-kube-api-access-n4pnk\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.169326 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.169302 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a88b42d3-8f3d-4847-861e-f33c7c04274d-data-volume\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.169529 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.169511 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a88b42d3-8f3d-4847-861e-f33c7c04274d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.170968 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.170940 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:29.171180 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.171162 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a88b42d3-8f3d-4847-861e-f33c7c04274d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.192915 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.192889 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4pnk\" (UniqueName: \"kubernetes.io/projected/a88b42d3-8f3d-4847-861e-f33c7c04274d-kube-api-access-n4pnk\") pod \"insights-runtime-extractor-9v4xs\" (UID: \"a88b42d3-8f3d-4847-861e-f33c7c04274d\") " pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.276313 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.276282 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9v4xs" May 20 21:44:29.346389 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.346361 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-2nbqz"] May 20 21:44:29.349380 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:29.349351 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda584980a_67ce_4416_b2d0_180c64eafacd.slice/crio-9d515e3619cd564a11b0494da86d0cf9a561036bc8764215c3ff151a7cc6646d WatchSource:0}: Error finding container 9d515e3619cd564a11b0494da86d0cf9a561036bc8764215c3ff151a7cc6646d: Status 404 returned error can't find the container with id 9d515e3619cd564a11b0494da86d0cf9a561036bc8764215c3ff151a7cc6646d May 20 21:44:29.427075 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.426997 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9v4xs"] May 20 21:44:29.430470 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:29.430447 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88b42d3_8f3d_4847_861e_f33c7c04274d.slice/crio-d134a3f42d12a726b34b94f23f044224936f231b83991f370ea65dcc93e113da WatchSource:0}: Error finding container d134a3f42d12a726b34b94f23f044224936f231b83991f370ea65dcc93e113da: Status 404 returned error can't find the container with id d134a3f42d12a726b34b94f23f044224936f231b83991f370ea65dcc93e113da May 20 21:44:29.875607 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.875562 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:44:29.881289 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.881260 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7eb15ff5-c1cf-40e2-a612-7e5494cc395e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-72ztf\" (UID: \"7eb15ff5-c1cf-40e2-a612-7e5494cc395e\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:44:29.977641 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.977602 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:44:29.980508 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.980484 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/09b71b40-b58b-4831-ac46-0f4c09f6f287-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-9gf48\" (UID: \"09b71b40-b58b-4831-ac46-0f4c09f6f287\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:44:29.989400 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.989373 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fx8b2\"" May 20 21:44:29.992466 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:29.992413 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" May 20 21:44:30.078361 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.078325 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:30.078538 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.078380 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:30.079276 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.079253 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/150654bb-1c66-4f64-97dd-5a1cdd4d370f-service-ca-bundle\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:30.081307 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.081280 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/150654bb-1c66-4f64-97dd-5a1cdd4d370f-metrics-certs\") pod \"router-default-58f97b448b-g9d6k\" (UID: \"150654bb-1c66-4f64-97dd-5a1cdd4d370f\") " pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:30.180096 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.179902 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hjt5b\"" May 20 21:44:30.184249 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.184179 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9v4xs" event={"ID":"a88b42d3-8f3d-4847-861e-f33c7c04274d","Type":"ContainerStarted","Data":"565a7d97d7c5175d0b3b90eac6abb22103953fbf55f35079ed3f1fae012a6650"} May 20 21:44:30.184249 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.184217 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9v4xs" event={"ID":"a88b42d3-8f3d-4847-861e-f33c7c04274d","Type":"ContainerStarted","Data":"d134a3f42d12a726b34b94f23f044224936f231b83991f370ea65dcc93e113da"} May 20 21:44:30.186122 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.186091 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" May 20 21:44:30.186490 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.186449 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-2nbqz" event={"ID":"a584980a-67ce-4416-b2d0-180c64eafacd","Type":"ContainerStarted","Data":"9d515e3619cd564a11b0494da86d0cf9a561036bc8764215c3ff151a7cc6646d"} May 20 21:44:30.255833 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.255281 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48"] May 20 21:44:30.267367 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:30.267331 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b71b40_b58b_4831_ac46_0f4c09f6f287.slice/crio-c7ee09a1494fdef76213254023bac26ee12ee84fc380b3e07a3b733b23326043 WatchSource:0}: Error finding container c7ee09a1494fdef76213254023bac26ee12ee84fc380b3e07a3b733b23326043: Status 404 returned error can't find the container with id c7ee09a1494fdef76213254023bac26ee12ee84fc380b3e07a3b733b23326043 May 20 21:44:30.367528 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.367502 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7vfkk\"" May 20 21:44:30.370666 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.370642 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:30.387443 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.387362 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf"] May 20 21:44:30.390247 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:30.390205 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eb15ff5_c1cf_40e2_a612_7e5494cc395e.slice/crio-6afa0bf2267d8b841fb5b83023722022eb4a6ba12a54e4010eb3587571ac7640 WatchSource:0}: Error finding container 6afa0bf2267d8b841fb5b83023722022eb4a6ba12a54e4010eb3587571ac7640: Status 404 returned error can't find the container with id 6afa0bf2267d8b841fb5b83023722022eb4a6ba12a54e4010eb3587571ac7640 May 20 21:44:30.525517 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:30.525480 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58f97b448b-g9d6k"] May 20 21:44:30.529241 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:30.529217 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150654bb_1c66_4f64_97dd_5a1cdd4d370f.slice/crio-358815d4b10ef6cc827448970d2389b3f90c5ea438e5ade0b5bfa6f46a6d3a71 WatchSource:0}: Error finding container 358815d4b10ef6cc827448970d2389b3f90c5ea438e5ade0b5bfa6f46a6d3a71: Status 404 returned error can't find the container with id 358815d4b10ef6cc827448970d2389b3f90c5ea438e5ade0b5bfa6f46a6d3a71 May 20 21:44:31.149881 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.148468 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bddb4c76c-hj7hk"] May 20 21:44:31.151173 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.151150 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.157025 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.157002 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 20 21:44:31.160097 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.159739 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 20 21:44:31.161030 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.160849 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 20 21:44:31.162780 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.162760 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 20 21:44:31.163825 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.163802 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dl8fd\"" May 20 21:44:31.165986 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.165750 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 20 21:44:31.181386 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.181356 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bddb4c76c-hj7hk"] May 20 21:44:31.194187 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.194124 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9v4xs" event={"ID":"a88b42d3-8f3d-4847-861e-f33c7c04274d","Type":"ContainerStarted","Data":"7a49251de39da06bf6a5cf585edce156bbb9fd6f255b82fa8be701e037c17dda"} May 20 21:44:31.197344 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.197317 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58f97b448b-g9d6k" event={"ID":"150654bb-1c66-4f64-97dd-5a1cdd4d370f","Type":"ContainerStarted","Data":"3b87a5cc1c718dc743a29d19740ca1d09803b1cfd4828caa024997130fe9be6b"} May 20 21:44:31.197455 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.197355 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58f97b448b-g9d6k" event={"ID":"150654bb-1c66-4f64-97dd-5a1cdd4d370f","Type":"ContainerStarted","Data":"358815d4b10ef6cc827448970d2389b3f90c5ea438e5ade0b5bfa6f46a6d3a71"} May 20 21:44:31.199324 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.199301 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" event={"ID":"09b71b40-b58b-4831-ac46-0f4c09f6f287","Type":"ContainerStarted","Data":"c7ee09a1494fdef76213254023bac26ee12ee84fc380b3e07a3b733b23326043"} May 20 21:44:31.201508 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.201455 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" event={"ID":"7eb15ff5-c1cf-40e2-a612-7e5494cc395e","Type":"ContainerStarted","Data":"6afa0bf2267d8b841fb5b83023722022eb4a6ba12a54e4010eb3587571ac7640"} May 20 21:44:31.273395 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.273189 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-58f97b448b-g9d6k" podStartSLOduration=33.273169642 podStartE2EDuration="33.273169642s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:44:31.272835856 +0000 UTC m=+86.054269107" watchObservedRunningTime="2026-05-20 21:44:31.273169642 +0000 UTC m=+86.054602897" May 20 21:44:31.289448 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.289389 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-serving-cert\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.289614 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.289487 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-service-ca\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.289614 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.289519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-oauth-config\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.289746 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.289565 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-console-config\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.289746 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.289656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-oauth-serving-cert\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.289851 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.289770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfm2k\" (UniqueName: \"kubernetes.io/projected/52ae6d02-ca60-4984-8c00-7618adaab88e-kube-api-access-pfm2k\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.371892 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.371859 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:31.374972 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.374937 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:31.391183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.391122 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pfm2k\" (UniqueName: \"kubernetes.io/projected/52ae6d02-ca60-4984-8c00-7618adaab88e-kube-api-access-pfm2k\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.391518 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-serving-cert\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.391587 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-service-ca\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.391616 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-oauth-config\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.391657 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-console-config\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.391705 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-oauth-serving-cert\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.392382 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-service-ca\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.392392 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-oauth-serving-cert\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.392963 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.392873 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-console-config\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.396456 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.396436 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-oauth-config\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.404137 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.404066 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-serving-cert\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.412335 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.412305 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfm2k\" (UniqueName: \"kubernetes.io/projected/52ae6d02-ca60-4984-8c00-7618adaab88e-kube-api-access-pfm2k\") pod \"console-7bddb4c76c-hj7hk\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:31.464228 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:31.464189 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:32.204409 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:32.204378 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:32.206160 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:32.205935 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58f97b448b-g9d6k" May 20 21:44:32.863178 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:32.862924 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bddb4c76c-hj7hk"] May 20 21:44:32.866647 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:32.866609 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ae6d02_ca60_4984_8c00_7618adaab88e.slice/crio-af00ac676371b8798e5d78cd7923d9f3895c6253820584a252dcf15564292950 WatchSource:0}: Error finding container af00ac676371b8798e5d78cd7923d9f3895c6253820584a252dcf15564292950: Status 404 returned error can't find the container with id af00ac676371b8798e5d78cd7923d9f3895c6253820584a252dcf15564292950 May 20 21:44:33.209115 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.209046 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" event={"ID":"09b71b40-b58b-4831-ac46-0f4c09f6f287","Type":"ContainerStarted","Data":"40646cd10d652be7deb453ef2e6bf9660087422dc003267217f6b4d86619eec8"} May 20 21:44:33.210527 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.210496 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bddb4c76c-hj7hk" event={"ID":"52ae6d02-ca60-4984-8c00-7618adaab88e","Type":"ContainerStarted","Data":"af00ac676371b8798e5d78cd7923d9f3895c6253820584a252dcf15564292950"} May 20 21:44:33.212038 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.212003 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" event={"ID":"7eb15ff5-c1cf-40e2-a612-7e5494cc395e","Type":"ContainerStarted","Data":"9ba65d7c7f6a1adc57a12cf63ca8f82ea8d138d1507d5e9fd835b75a32f0dd52"} May 20 21:44:33.214344 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.214315 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9v4xs" event={"ID":"a88b42d3-8f3d-4847-861e-f33c7c04274d","Type":"ContainerStarted","Data":"7fceb8914fb279184aaf705f08bb0056849c3d18a6cbfe5369c1bb7fba0c43a8"} May 20 21:44:33.230067 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.230008 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-9gf48" podStartSLOduration=32.808505514 podStartE2EDuration="35.229994976s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:44:30.26971373 +0000 UTC m=+85.051146963" lastFinishedPulling="2026-05-20 21:44:32.691203182 +0000 UTC m=+87.472636425" observedRunningTime="2026-05-20 21:44:33.228266066 +0000 UTC m=+88.009699317" watchObservedRunningTime="2026-05-20 21:44:33.229994976 +0000 UTC m=+88.011428228" May 20 21:44:33.251926 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.251867 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9v4xs" podStartSLOduration=2.041470612 podStartE2EDuration="5.251849101s" podCreationTimestamp="2026-05-20 21:44:28 +0000 UTC" firstStartedPulling="2026-05-20 21:44:29.479149622 +0000 UTC m=+84.260582851" lastFinishedPulling="2026-05-20 21:44:32.689528093 +0000 UTC m=+87.470961340" observedRunningTime="2026-05-20 21:44:33.251236496 +0000 UTC m=+88.032669748" watchObservedRunningTime="2026-05-20 21:44:33.251849101 +0000 UTC m=+88.033282352" May 20 21:44:33.275247 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.275186 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-72ztf" podStartSLOduration=32.971778786 podStartE2EDuration="35.275168243s" podCreationTimestamp="2026-05-20 21:43:58 +0000 UTC" firstStartedPulling="2026-05-20 21:44:30.39232463 +0000 UTC m=+85.173757865" lastFinishedPulling="2026-05-20 21:44:32.695714088 +0000 UTC m=+87.477147322" observedRunningTime="2026-05-20 21:44:33.274630871 +0000 UTC m=+88.056064122" watchObservedRunningTime="2026-05-20 21:44:33.275168243 +0000 UTC m=+88.056601498" May 20 21:44:33.377619 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.377586 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr"] May 20 21:44:33.391519 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.391138 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:33.394624 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.394599 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6rltd\"" May 20 21:44:33.394950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.394912 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" May 20 21:44:33.398246 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.398203 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr"] May 20 21:44:33.411050 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.410981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df9c042-0aaf-4533-ab9e-7c72222a641f-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr\" (UID: \"4df9c042-0aaf-4533-ab9e-7c72222a641f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:33.511872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.511785 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df9c042-0aaf-4533-ab9e-7c72222a641f-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr\" (UID: \"4df9c042-0aaf-4533-ab9e-7c72222a641f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:33.514945 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.514914 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4df9c042-0aaf-4533-ab9e-7c72222a641f-tls-certificates\") pod \"prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr\" (UID: \"4df9c042-0aaf-4533-ab9e-7c72222a641f\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:33.704813 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.704707 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:33.881045 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:33.881002 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr"] May 20 21:44:33.886239 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:33.886179 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df9c042_0aaf_4533_ab9e_7c72222a641f.slice/crio-f8986ab6b5b634ffafb6695363e3c8ae5c28f30925451e494075da9cda43d481 WatchSource:0}: Error finding container f8986ab6b5b634ffafb6695363e3c8ae5c28f30925451e494075da9cda43d481: Status 404 returned error can't find the container with id f8986ab6b5b634ffafb6695363e3c8ae5c28f30925451e494075da9cda43d481 May 20 21:44:34.218710 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:34.218671 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" event={"ID":"4df9c042-0aaf-4533-ab9e-7c72222a641f","Type":"ContainerStarted","Data":"f8986ab6b5b634ffafb6695363e3c8ae5c28f30925451e494075da9cda43d481"} May 20 21:44:37.230879 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:37.230842 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" event={"ID":"4df9c042-0aaf-4533-ab9e-7c72222a641f","Type":"ContainerStarted","Data":"ecaab5ed6f479fe132177fd45edbe375a392f5b4db2c4b368e0adcea601a07d1"} May 20 21:44:37.231373 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:37.231094 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:37.232714 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:37.232688 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bddb4c76c-hj7hk" event={"ID":"52ae6d02-ca60-4984-8c00-7618adaab88e","Type":"ContainerStarted","Data":"18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408"} May 20 21:44:37.236173 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:37.236150 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" May 20 21:44:37.254840 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:37.254794 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr" podStartSLOduration=1.345281447 podStartE2EDuration="4.25478327s" podCreationTimestamp="2026-05-20 21:44:33 +0000 UTC" firstStartedPulling="2026-05-20 21:44:33.888910316 +0000 UTC m=+88.670343549" lastFinishedPulling="2026-05-20 21:44:36.798412135 +0000 UTC m=+91.579845372" observedRunningTime="2026-05-20 21:44:37.253009962 +0000 UTC m=+92.034443213" watchObservedRunningTime="2026-05-20 21:44:37.25478327 +0000 UTC m=+92.036216568" May 20 21:44:41.465429 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:41.465343 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:41.465429 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:41.465415 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:44:41.466605 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:41.466563 2565 patch_prober.go:28] interesting pod/console-7bddb4c76c-hj7hk container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" start-of-body= May 20 21:44:41.466730 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:41.466644 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7bddb4c76c-hj7hk" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerName="console" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" May 20 21:44:42.823470 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.823418 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7bddb4c76c-hj7hk" podStartSLOduration=7.896703526 podStartE2EDuration="11.823404823s" podCreationTimestamp="2026-05-20 21:44:31 +0000 UTC" firstStartedPulling="2026-05-20 21:44:32.868906553 +0000 UTC m=+87.650339785" lastFinishedPulling="2026-05-20 21:44:36.795607837 +0000 UTC m=+91.577041082" observedRunningTime="2026-05-20 21:44:37.29284445 +0000 UTC m=+92.074277703" watchObservedRunningTime="2026-05-20 21:44:42.823404823 +0000 UTC m=+97.604838108" May 20 21:44:42.824089 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.824071 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vxm65"] May 20 21:44:42.827183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.827164 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.830791 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.830586 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mbsh8\"" May 20 21:44:42.830791 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.830730 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 20 21:44:42.832454 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.832340 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 20 21:44:42.832872 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.832703 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 20 21:44:42.833198 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.833178 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 20 21:44:42.901746 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901707 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/646bd61c-fc09-4da3-b804-f6947b5e451f-metrics-client-ca\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.901913 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901774 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-root\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.901913 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv225\" (UniqueName: \"kubernetes.io/projected/646bd61c-fc09-4da3-b804-f6947b5e451f-kube-api-access-fv225\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.901913 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901845 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-sys\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.901913 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901893 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-accelerators-collector-config\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.902152 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-wtmp\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.902152 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.901981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.902152 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.902013 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-tls\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:42.902152 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:42.902043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-textfile\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003188 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003158 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-textfile\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003230 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/646bd61c-fc09-4da3-b804-f6947b5e451f-metrics-client-ca\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003273 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-root\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003296 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fv225\" (UniqueName: \"kubernetes.io/projected/646bd61c-fc09-4da3-b804-f6947b5e451f-kube-api-access-fv225\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003367 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003338 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-sys\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003613 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003414 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-root\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003613 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003495 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-accelerators-collector-config\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003613 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003549 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-textfile\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003613 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003598 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-wtmp\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003813 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003627 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-sys\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003813 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003627 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.003813 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003683 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-tls\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.004183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.003897 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/646bd61c-fc09-4da3-b804-f6947b5e451f-metrics-client-ca\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.004183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.004031 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-wtmp\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.004359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.004265 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-accelerators-collector-config\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.007459 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.007435 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-tls\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.007708 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.007664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/646bd61c-fc09-4da3-b804-f6947b5e451f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.012346 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.012318 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv225\" (UniqueName: \"kubernetes.io/projected/646bd61c-fc09-4da3-b804-f6947b5e451f-kube-api-access-fv225\") pod \"node-exporter-vxm65\" (UID: \"646bd61c-fc09-4da3-b804-f6947b5e451f\") " pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:43.132436 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.132358 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xj72f" May 20 21:44:43.143141 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:43.143102 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vxm65" May 20 21:44:44.619674 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.619631 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:44:44.620132 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.619711 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:44:44.622604 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.622561 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1eddfa4-fc63-448b-b6f9-2e321d8b02f2-metrics-tls\") pod \"dns-default-6xhk7\" (UID: \"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2\") " pod="openshift-dns/dns-default-6xhk7" May 20 21:44:44.622750 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.622730 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"image-registry-d49b96d4b-jdts2\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:44:44.720260 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.720226 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:44:44.722979 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.722955 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b-cert\") pod \"ingress-canary-6hfxx\" (UID: \"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b\") " pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:44:44.771637 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.771610 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7vrzg\"" May 20 21:44:44.779991 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.779969 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:44:44.849071 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.849038 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ttsl8\"" May 20 21:44:44.856479 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.856450 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6xhk7" May 20 21:44:44.856942 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.856904 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wdrst\"" May 20 21:44:44.865202 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:44.865183 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6hfxx" May 20 21:44:46.923630 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:46.923554 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646bd61c_fc09_4da3_b804_f6947b5e451f.slice/crio-3efa0b36804bbf1f915b9eb7b1764f15e14d3fe7943b18e2f3dd2bc2be75a9a1 WatchSource:0}: Error finding container 3efa0b36804bbf1f915b9eb7b1764f15e14d3fe7943b18e2f3dd2bc2be75a9a1: Status 404 returned error can't find the container with id 3efa0b36804bbf1f915b9eb7b1764f15e14d3fe7943b18e2f3dd2bc2be75a9a1 May 20 21:44:47.079051 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.079026 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6xhk7"] May 20 21:44:47.081680 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:47.081649 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1eddfa4_fc63_448b_b6f9_2e321d8b02f2.slice/crio-56da8e2d4e16126251b00cd664a121f264fc934850f7f1aafb7a11776a67ecf2 WatchSource:0}: Error finding container 56da8e2d4e16126251b00cd664a121f264fc934850f7f1aafb7a11776a67ecf2: Status 404 returned error can't find the container with id 56da8e2d4e16126251b00cd664a121f264fc934850f7f1aafb7a11776a67ecf2 May 20 21:44:47.271609 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.271484 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xhk7" event={"ID":"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2","Type":"ContainerStarted","Data":"56da8e2d4e16126251b00cd664a121f264fc934850f7f1aafb7a11776a67ecf2"} May 20 21:44:47.273817 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.273780 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vxm65" event={"ID":"646bd61c-fc09-4da3-b804-f6947b5e451f","Type":"ContainerStarted","Data":"3efa0b36804bbf1f915b9eb7b1764f15e14d3fe7943b18e2f3dd2bc2be75a9a1"} May 20 21:44:47.275546 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.275433 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-2nbqz" event={"ID":"a584980a-67ce-4416-b2d0-180c64eafacd","Type":"ContainerStarted","Data":"c527890badb39083adb71b1eb48e4d6e98b08ae0e542cfd6bf38aa4d22929b4b"} May 20 21:44:47.276309 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.276283 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:47.281018 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.280985 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-2nbqz" May 20 21:44:47.304015 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.303958 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-2nbqz" podStartSLOduration=1.661572346 podStartE2EDuration="19.303941185s" podCreationTimestamp="2026-05-20 21:44:28 +0000 UTC" firstStartedPulling="2026-05-20 21:44:29.351515856 +0000 UTC m=+84.132949092" lastFinishedPulling="2026-05-20 21:44:46.993884686 +0000 UTC m=+101.775317931" observedRunningTime="2026-05-20 21:44:47.301776852 +0000 UTC m=+102.083210118" watchObservedRunningTime="2026-05-20 21:44:47.303941185 +0000 UTC m=+102.085374437" May 20 21:44:47.317267 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.317017 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6hfxx"] May 20 21:44:47.317267 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:47.317054 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d49b96d4b-jdts2"] May 20 21:44:47.318367 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:47.318336 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7dbaa90_54fe_4da4_904c_e6f6c35d9e89.slice/crio-3da5a25e5ab6225c863cb7c091ee768a0af105ecb3ff875d804713b5f02cc179 WatchSource:0}: Error finding container 3da5a25e5ab6225c863cb7c091ee768a0af105ecb3ff875d804713b5f02cc179: Status 404 returned error can't find the container with id 3da5a25e5ab6225c863cb7c091ee768a0af105ecb3ff875d804713b5f02cc179 May 20 21:44:47.319867 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:44:47.319169 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5d97ad_7543_4fd6_aabb_7e6a4e1c9e7b.slice/crio-8bc31ccce1d02a224f63cfc42dcd0305cedc919d4d4027bffb819f9f8d248457 WatchSource:0}: Error finding container 8bc31ccce1d02a224f63cfc42dcd0305cedc919d4d4027bffb819f9f8d248457: Status 404 returned error can't find the container with id 8bc31ccce1d02a224f63cfc42dcd0305cedc919d4d4027bffb819f9f8d248457 May 20 21:44:48.281289 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:48.281236 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hfxx" event={"ID":"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b","Type":"ContainerStarted","Data":"8bc31ccce1d02a224f63cfc42dcd0305cedc919d4d4027bffb819f9f8d248457"} May 20 21:44:48.285438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:48.285316 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" event={"ID":"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89","Type":"ContainerStarted","Data":"bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e"} May 20 21:44:48.285438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:48.285356 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" event={"ID":"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89","Type":"ContainerStarted","Data":"3da5a25e5ab6225c863cb7c091ee768a0af105ecb3ff875d804713b5f02cc179"} May 20 21:44:48.285438 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:48.285403 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:44:48.316984 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:48.316886 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" podStartSLOduration=102.316868934 podStartE2EDuration="1m42.316868934s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:44:48.313351463 +0000 UTC m=+103.094784723" watchObservedRunningTime="2026-05-20 21:44:48.316868934 +0000 UTC m=+103.098302186" May 20 21:44:51.297476 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.297431 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xhk7" event={"ID":"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2","Type":"ContainerStarted","Data":"4f23fb0b6382ef16e7101c41680243014fad3627540e96cc9a994c99759c619b"} May 20 21:44:51.297476 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.297482 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6xhk7" event={"ID":"d1eddfa4-fc63-448b-b6f9-2e321d8b02f2","Type":"ContainerStarted","Data":"97e0642ee5bf8e378fd85976e1f5e7f7ac8848f73ea44f41e931cf236bef874c"} May 20 21:44:51.298072 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.297618 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6xhk7" May 20 21:44:51.299225 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.299191 2565 generic.go:358] "Generic (PLEG): container finished" podID="646bd61c-fc09-4da3-b804-f6947b5e451f" containerID="906d02c2d90cfb09e1191d9fa6f72c4280b065cacb5eeaf2b52e5e174e43c8a9" exitCode=0 May 20 21:44:51.299336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.299266 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vxm65" event={"ID":"646bd61c-fc09-4da3-b804-f6947b5e451f","Type":"ContainerDied","Data":"906d02c2d90cfb09e1191d9fa6f72c4280b065cacb5eeaf2b52e5e174e43c8a9"} May 20 21:44:51.301170 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.301118 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6hfxx" event={"ID":"7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b","Type":"ContainerStarted","Data":"7497e677050402c1ef425dc12e98c08b79a923536218ef23c6847bf01887a134"} May 20 21:44:51.326655 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.326602 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-6xhk7" podStartSLOduration=68.142817548 podStartE2EDuration="1m11.32656449s" podCreationTimestamp="2026-05-20 21:43:40 +0000 UTC" firstStartedPulling="2026-05-20 21:44:47.083532478 +0000 UTC m=+101.864965707" lastFinishedPulling="2026-05-20 21:44:50.26727942 +0000 UTC m=+105.048712649" observedRunningTime="2026-05-20 21:44:51.324784028 +0000 UTC m=+106.106217279" watchObservedRunningTime="2026-05-20 21:44:51.32656449 +0000 UTC m=+106.107997745" May 20 21:44:51.345760 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.345714 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6hfxx" podStartSLOduration=68.396088402 podStartE2EDuration="1m11.345701054s" podCreationTimestamp="2026-05-20 21:43:40 +0000 UTC" firstStartedPulling="2026-05-20 21:44:47.322997212 +0000 UTC m=+102.104430444" lastFinishedPulling="2026-05-20 21:44:50.272609867 +0000 UTC m=+105.054043096" observedRunningTime="2026-05-20 21:44:51.344286935 +0000 UTC m=+106.125720197" watchObservedRunningTime="2026-05-20 21:44:51.345701054 +0000 UTC m=+106.127134306" May 20 21:44:51.465585 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.465544 2565 patch_prober.go:28] interesting pod/console-7bddb4c76c-hj7hk container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" start-of-body= May 20 21:44:51.465726 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.465608 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7bddb4c76c-hj7hk" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerName="console" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" May 20 21:44:51.539348 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:51.539309 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bddb4c76c-hj7hk"] May 20 21:44:52.306420 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:52.306376 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vxm65" event={"ID":"646bd61c-fc09-4da3-b804-f6947b5e451f","Type":"ContainerStarted","Data":"97346c606535efd232f08f0b91040e5b50f3e501008aaa3d8f32771abc0e8aca"} May 20 21:44:52.306420 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:52.306423 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vxm65" event={"ID":"646bd61c-fc09-4da3-b804-f6947b5e451f","Type":"ContainerStarted","Data":"3993e7b7ec2c97ec8645ce9e244b61bfdcb2c9e9d6542b5473f31eb496e78d78"} May 20 21:44:56.412658 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:56.412598 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vxm65" podStartSLOduration=11.097861988 podStartE2EDuration="14.412566659s" podCreationTimestamp="2026-05-20 21:44:42 +0000 UTC" firstStartedPulling="2026-05-20 21:44:46.926711091 +0000 UTC m=+101.708144322" lastFinishedPulling="2026-05-20 21:44:50.241415747 +0000 UTC m=+105.022848993" observedRunningTime="2026-05-20 21:44:52.329905033 +0000 UTC m=+107.111338286" watchObservedRunningTime="2026-05-20 21:44:56.412566659 +0000 UTC m=+111.193999909" May 20 21:44:56.413564 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:44:56.413541 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d49b96d4b-jdts2"] May 20 21:45:01.309180 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:01.309128 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6xhk7" May 20 21:45:06.419336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:06.419306 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:45:10.901847 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:10.901806 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" podUID="33defec2-3d29-4850-b9b7-fafe27a4225a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 20 21:45:15.598352 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:15.598315 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:45:15.600603 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:15.600581 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5df8c31-3ae1-4ae3-9a87-3da35016fd6f-metrics-certs\") pod \"network-metrics-daemon-2slgv\" (UID: \"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f\") " pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:45:15.660055 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:15.660029 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-p8dpp\"" May 20 21:45:15.667982 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:15.667961 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2slgv" May 20 21:45:15.801732 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:15.801706 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2slgv"] May 20 21:45:15.804061 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:45:15.804032 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5df8c31_3ae1_4ae3_9a87_3da35016fd6f.slice/crio-dfab671d6345c12dac17502887048af47081bdceb30d432731cfb76b9401000f WatchSource:0}: Error finding container dfab671d6345c12dac17502887048af47081bdceb30d432731cfb76b9401000f: Status 404 returned error can't find the container with id dfab671d6345c12dac17502887048af47081bdceb30d432731cfb76b9401000f May 20 21:45:16.379190 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:16.379152 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2slgv" event={"ID":"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f","Type":"ContainerStarted","Data":"dfab671d6345c12dac17502887048af47081bdceb30d432731cfb76b9401000f"} May 20 21:45:16.561648 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:16.561531 2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7bddb4c76c-hj7hk" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerName="console" containerID="cri-o://18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408" gracePeriod=15 May 20 21:45:17.117787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.117765 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bddb4c76c-hj7hk_52ae6d02-ca60-4984-8c00-7618adaab88e/console/0.log" May 20 21:45:17.118062 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.117837 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:45:17.212155 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212124 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-service-ca\") pod \"52ae6d02-ca60-4984-8c00-7618adaab88e\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " May 20 21:45:17.212336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212173 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-oauth-config\") pod \"52ae6d02-ca60-4984-8c00-7618adaab88e\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " May 20 21:45:17.212336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212201 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-oauth-serving-cert\") pod \"52ae6d02-ca60-4984-8c00-7618adaab88e\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " May 20 21:45:17.212336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212236 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-serving-cert\") pod \"52ae6d02-ca60-4984-8c00-7618adaab88e\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " May 20 21:45:17.212466 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212415 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfm2k\" (UniqueName: \"kubernetes.io/projected/52ae6d02-ca60-4984-8c00-7618adaab88e-kube-api-access-pfm2k\") pod \"52ae6d02-ca60-4984-8c00-7618adaab88e\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " May 20 21:45:17.212518 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212479 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-console-config\") pod \"52ae6d02-ca60-4984-8c00-7618adaab88e\" (UID: \"52ae6d02-ca60-4984-8c00-7618adaab88e\") " May 20 21:45:17.212651 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212621 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-service-ca" (OuterVolumeSpecName: "service-ca") pod "52ae6d02-ca60-4984-8c00-7618adaab88e" (UID: "52ae6d02-ca60-4984-8c00-7618adaab88e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 21:45:17.212789 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212763 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "52ae6d02-ca60-4984-8c00-7618adaab88e" (UID: "52ae6d02-ca60-4984-8c00-7618adaab88e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 21:45:17.212854 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212818 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-console-config" (OuterVolumeSpecName: "console-config") pod "52ae6d02-ca60-4984-8c00-7618adaab88e" (UID: "52ae6d02-ca60-4984-8c00-7618adaab88e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 21:45:17.212930 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212913 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-service-ca\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:17.212977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212936 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-oauth-serving-cert\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:17.212977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.212950 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/52ae6d02-ca60-4984-8c00-7618adaab88e-console-config\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:17.214538 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.214521 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "52ae6d02-ca60-4984-8c00-7618adaab88e" (UID: "52ae6d02-ca60-4984-8c00-7618adaab88e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 20 21:45:17.214768 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.214748 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "52ae6d02-ca60-4984-8c00-7618adaab88e" (UID: "52ae6d02-ca60-4984-8c00-7618adaab88e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 20 21:45:17.214844 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.214770 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52ae6d02-ca60-4984-8c00-7618adaab88e-kube-api-access-pfm2k" (OuterVolumeSpecName: "kube-api-access-pfm2k") pod "52ae6d02-ca60-4984-8c00-7618adaab88e" (UID: "52ae6d02-ca60-4984-8c00-7618adaab88e"). InnerVolumeSpecName "kube-api-access-pfm2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 21:45:17.313254 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.313228 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfm2k\" (UniqueName: \"kubernetes.io/projected/52ae6d02-ca60-4984-8c00-7618adaab88e-kube-api-access-pfm2k\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:17.313254 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.313253 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-oauth-config\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:17.313378 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.313264 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/52ae6d02-ca60-4984-8c00-7618adaab88e-console-serving-cert\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:17.382999 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.382979 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7bddb4c76c-hj7hk_52ae6d02-ca60-4984-8c00-7618adaab88e/console/0.log" May 20 21:45:17.383119 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.383014 2565 generic.go:358] "Generic (PLEG): container finished" podID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerID="18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408" exitCode=2 May 20 21:45:17.383119 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.383083 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bddb4c76c-hj7hk" event={"ID":"52ae6d02-ca60-4984-8c00-7618adaab88e","Type":"ContainerDied","Data":"18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408"} May 20 21:45:17.383119 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.383093 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bddb4c76c-hj7hk" May 20 21:45:17.383119 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.383112 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bddb4c76c-hj7hk" event={"ID":"52ae6d02-ca60-4984-8c00-7618adaab88e","Type":"ContainerDied","Data":"af00ac676371b8798e5d78cd7923d9f3895c6253820584a252dcf15564292950"} May 20 21:45:17.383306 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.383127 2565 scope.go:117] "RemoveContainer" containerID="18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408" May 20 21:45:17.391446 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.391425 2565 scope.go:117] "RemoveContainer" containerID="18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408" May 20 21:45:17.391684 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:45:17.391667 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408\": container with ID starting with 18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408 not found: ID does not exist" containerID="18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408" May 20 21:45:17.391749 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.391691 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408"} err="failed to get container status \"18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408\": rpc error: code = NotFound desc = could not find container \"18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408\": container with ID starting with 18be9d6084e77bb8125b9347a763593491ef946a3c9ff49ea0fbaf1b923c7408 not found: ID does not exist" May 20 21:45:17.405661 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.405641 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bddb4c76c-hj7hk"] May 20 21:45:17.410173 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.410149 2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bddb4c76c-hj7hk"] May 20 21:45:17.745676 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:17.745645 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" path="/var/lib/kubelet/pods/52ae6d02-ca60-4984-8c00-7618adaab88e/volumes" May 20 21:45:18.387148 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:18.387104 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2slgv" event={"ID":"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f","Type":"ContainerStarted","Data":"8fff5dd3c759a243f93d48bc56d99c1a6f9afdaedd7bf433ef7d1f3f615fc466"} May 20 21:45:18.387148 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:18.387145 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2slgv" event={"ID":"a5df8c31-3ae1-4ae3-9a87-3da35016fd6f","Type":"ContainerStarted","Data":"07d44a562305428e7cd300417496d24c12d1c3b53f2cd7864f77f94b66e25ada"} May 20 21:45:18.405041 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:18.404982 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2slgv" podStartSLOduration=130.910618245 podStartE2EDuration="2m12.40496707s" podCreationTimestamp="2026-05-20 21:43:06 +0000 UTC" firstStartedPulling="2026-05-20 21:45:15.805693145 +0000 UTC m=+130.587126375" lastFinishedPulling="2026-05-20 21:45:17.300041967 +0000 UTC m=+132.081475200" observedRunningTime="2026-05-20 21:45:18.40288642 +0000 UTC m=+133.184319670" watchObservedRunningTime="2026-05-20 21:45:18.40496707 +0000 UTC m=+133.186400321" May 20 21:45:20.901255 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:20.901218 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" podUID="33defec2-3d29-4850-b9b7-fafe27a4225a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 20 21:45:21.432873 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.432833 2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" podUID="a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" containerName="registry" containerID="cri-o://bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e" gracePeriod=30 May 20 21:45:21.711365 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.711341 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:45:21.740437 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740411 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9j5t\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-kube-api-access-d9j5t\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740591 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740462 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-bound-sa-token\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740591 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740488 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-certificates\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740591 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740545 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740603 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-installation-pull-secrets\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740627 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-trusted-ca\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740654 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-ca-trust-extracted\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.740766 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.740685 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-image-registry-private-configuration\") pod \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\" (UID: \"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89\") " May 20 21:45:21.742938 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.741411 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 21:45:21.743130 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.743076 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 20 21:45:21.744227 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.744183 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 21:45:21.744320 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.744271 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 20 21:45:21.744792 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.744766 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-kube-api-access-d9j5t" (OuterVolumeSpecName: "kube-api-access-d9j5t") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "kube-api-access-d9j5t". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 21:45:21.744971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.744904 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 20 21:45:21.745058 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.745017 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 21:45:21.751520 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.751497 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" (UID: "a7dbaa90-54fe-4da4-904c-e6f6c35d9e89"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 20 21:45:21.841241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841209 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9j5t\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-kube-api-access-d9j5t\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841235 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-bound-sa-token\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841245 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-certificates\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841434 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841255 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-registry-tls\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841434 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841265 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-installation-pull-secrets\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841434 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841273 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-trusted-ca\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841434 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841283 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-ca-trust-extracted\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:21.841434 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:21.841293 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89-image-registry-private-configuration\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:45:22.400212 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.400180 2565 generic.go:358] "Generic (PLEG): container finished" podID="a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" containerID="bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e" exitCode=0 May 20 21:45:22.400721 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.400239 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" May 20 21:45:22.400721 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.400271 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" event={"ID":"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89","Type":"ContainerDied","Data":"bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e"} May 20 21:45:22.400721 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.400324 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d49b96d4b-jdts2" event={"ID":"a7dbaa90-54fe-4da4-904c-e6f6c35d9e89","Type":"ContainerDied","Data":"3da5a25e5ab6225c863cb7c091ee768a0af105ecb3ff875d804713b5f02cc179"} May 20 21:45:22.400721 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.400349 2565 scope.go:117] "RemoveContainer" containerID="bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e" May 20 21:45:22.408190 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.408173 2565 scope.go:117] "RemoveContainer" containerID="bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e" May 20 21:45:22.408433 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:45:22.408416 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e\": container with ID starting with bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e not found: ID does not exist" containerID="bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e" May 20 21:45:22.408476 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.408442 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e"} err="failed to get container status \"bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e\": rpc error: code = NotFound desc = could not find container \"bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e\": container with ID starting with bc9df5b2bf823aad85e9d0c1558b36540b31c884aa1ed675b26aabe2edba373e not found: ID does not exist" May 20 21:45:22.421129 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.421102 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d49b96d4b-jdts2"] May 20 21:45:22.430780 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:22.430759 2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d49b96d4b-jdts2"] May 20 21:45:23.745479 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:23.745450 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" path="/var/lib/kubelet/pods/a7dbaa90-54fe-4da4-904c-e6f6c35d9e89/volumes" May 20 21:45:25.410824 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:25.410786 2565 generic.go:358] "Generic (PLEG): container finished" podID="46a4c312-92f4-4a59-8acd-96a2c204d00c" containerID="a06301ac596af6d5a6e108b3288cf557afd06f1f746db31f92a6296f6410810e" exitCode=0 May 20 21:45:25.411224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:25.410853 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" event={"ID":"46a4c312-92f4-4a59-8acd-96a2c204d00c","Type":"ContainerDied","Data":"a06301ac596af6d5a6e108b3288cf557afd06f1f746db31f92a6296f6410810e"} May 20 21:45:25.411224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:25.411145 2565 scope.go:117] "RemoveContainer" containerID="a06301ac596af6d5a6e108b3288cf557afd06f1f746db31f92a6296f6410810e" May 20 21:45:25.412172 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:25.412149 2565 generic.go:358] "Generic (PLEG): container finished" podID="0260fbc2-244d-4d26-80f6-55febb670445" containerID="65d40b3391a31a4b3201b30c8ea177196fd697dee4190e284dd5857a877b5587" exitCode=0 May 20 21:45:25.412251 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:25.412226 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" event={"ID":"0260fbc2-244d-4d26-80f6-55febb670445","Type":"ContainerDied","Data":"65d40b3391a31a4b3201b30c8ea177196fd697dee4190e284dd5857a877b5587"} May 20 21:45:25.412536 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:25.412521 2565 scope.go:117] "RemoveContainer" containerID="65d40b3391a31a4b3201b30c8ea177196fd697dee4190e284dd5857a877b5587" May 20 21:45:26.416460 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:26.416419 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-tlvqp" event={"ID":"46a4c312-92f4-4a59-8acd-96a2c204d00c","Type":"ContainerStarted","Data":"ebb484ccd47094d1821f18f06fb1faf85e03df759aaaa9f8f9ccc1fd532e681c"} May 20 21:45:26.417864 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:26.417843 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-qq26p" event={"ID":"0260fbc2-244d-4d26-80f6-55febb670445","Type":"ContainerStarted","Data":"3bc271418f77c2e05b367f6073be104fe03ef81a311567b3afb700a44645b71c"} May 20 21:45:30.430298 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.430215 2565 generic.go:358] "Generic (PLEG): container finished" podID="55be9913-a333-4eef-90a1-c2b3311cf9a6" containerID="e365942d84c426f893afdc6fe486663c8245aee185ad3d024c56344d515c759d" exitCode=0 May 20 21:45:30.430763 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.430294 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" event={"ID":"55be9913-a333-4eef-90a1-c2b3311cf9a6","Type":"ContainerDied","Data":"e365942d84c426f893afdc6fe486663c8245aee185ad3d024c56344d515c759d"} May 20 21:45:30.430763 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.430713 2565 scope.go:117] "RemoveContainer" containerID="e365942d84c426f893afdc6fe486663c8245aee185ad3d024c56344d515c759d" May 20 21:45:30.901437 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.901392 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" podUID="33defec2-3d29-4850-b9b7-fafe27a4225a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 20 21:45:30.901633 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.901457 2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" May 20 21:45:30.901968 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.901949 2565 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"986a17bffef8d6231538ba57589a9dda6db0fd7b59af0302cfc41cf0049974a9"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 20 21:45:30.902038 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:30.901988 2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" podUID="33defec2-3d29-4850-b9b7-fafe27a4225a" containerName="service-proxy" containerID="cri-o://986a17bffef8d6231538ba57589a9dda6db0fd7b59af0302cfc41cf0049974a9" gracePeriod=30 May 20 21:45:31.435340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:31.435307 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-kxsw4" event={"ID":"55be9913-a333-4eef-90a1-c2b3311cf9a6","Type":"ContainerStarted","Data":"4cf096a75d86abcef85d72b44679b83eb3e5801e9ad8d79073a5fce2474fd3f2"} May 20 21:45:31.437249 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:31.437223 2565 generic.go:358] "Generic (PLEG): container finished" podID="33defec2-3d29-4850-b9b7-fafe27a4225a" containerID="986a17bffef8d6231538ba57589a9dda6db0fd7b59af0302cfc41cf0049974a9" exitCode=2 May 20 21:45:31.437364 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:31.437289 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" event={"ID":"33defec2-3d29-4850-b9b7-fafe27a4225a","Type":"ContainerDied","Data":"986a17bffef8d6231538ba57589a9dda6db0fd7b59af0302cfc41cf0049974a9"} May 20 21:45:31.437364 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:45:31.437318 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6f9c99bc8c-498wm" event={"ID":"33defec2-3d29-4850-b9b7-fafe27a4225a","Type":"ContainerStarted","Data":"89a7ddda996a40993fef21e79236ab1bc9b78cb9dcab1b1c68c8e8de4068b41f"} May 20 21:48:00.671773 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.671734 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk"] May 20 21:48:00.672206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.672081 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerName="console" May 20 21:48:00.672206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.672096 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerName="console" May 20 21:48:00.672206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.672112 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" containerName="registry" May 20 21:48:00.672206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.672117 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" containerName="registry" May 20 21:48:00.672206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.672176 2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="52ae6d02-ca60-4984-8c00-7618adaab88e" containerName="console" May 20 21:48:00.672206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.672189 2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="a7dbaa90-54fe-4da4-904c-e6f6c35d9e89" containerName="registry" May 20 21:48:00.673963 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.673946 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.676812 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.676786 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 20 21:48:00.677067 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.677048 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 20 21:48:00.677181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.677053 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 20 21:48:00.677181 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.677112 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-t75sx\"" May 20 21:48:00.677394 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.677380 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 20 21:48:00.689025 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.689004 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk"] May 20 21:48:00.793506 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.793478 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa465369-893a-4dce-8da9-1029d174b580-webhook-cert\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.793688 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.793525 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa465369-893a-4dce-8da9-1029d174b580-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.793688 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.793559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qqdg\" (UniqueName: \"kubernetes.io/projected/fa465369-893a-4dce-8da9-1029d174b580-kube-api-access-6qqdg\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.893946 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.893912 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa465369-893a-4dce-8da9-1029d174b580-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.893946 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.893957 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6qqdg\" (UniqueName: \"kubernetes.io/projected/fa465369-893a-4dce-8da9-1029d174b580-kube-api-access-6qqdg\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.894233 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.894212 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa465369-893a-4dce-8da9-1029d174b580-webhook-cert\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.896398 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.896377 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fa465369-893a-4dce-8da9-1029d174b580-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.896495 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.896412 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fa465369-893a-4dce-8da9-1029d174b580-webhook-cert\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.903817 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.903786 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qqdg\" (UniqueName: \"kubernetes.io/projected/fa465369-893a-4dce-8da9-1029d174b580-kube-api-access-6qqdg\") pod \"opendatahub-operator-controller-manager-7967c8df76-6pghk\" (UID: \"fa465369-893a-4dce-8da9-1029d174b580\") " pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:00.984495 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:00.984440 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:01.119199 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:01.119164 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk"] May 20 21:48:01.122435 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:48:01.122407 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa465369_893a_4dce_8da9_1029d174b580.slice/crio-d64a9c78c00c108af5b6a2ad784a81711b9043a1ff5661e363bea3baa9ca0b45 WatchSource:0}: Error finding container d64a9c78c00c108af5b6a2ad784a81711b9043a1ff5661e363bea3baa9ca0b45: Status 404 returned error can't find the container with id d64a9c78c00c108af5b6a2ad784a81711b9043a1ff5661e363bea3baa9ca0b45 May 20 21:48:01.867382 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:01.867337 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" event={"ID":"fa465369-893a-4dce-8da9-1029d174b580","Type":"ContainerStarted","Data":"d64a9c78c00c108af5b6a2ad784a81711b9043a1ff5661e363bea3baa9ca0b45"} May 20 21:48:03.874873 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:03.874835 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" event={"ID":"fa465369-893a-4dce-8da9-1029d174b580","Type":"ContainerStarted","Data":"ee1c8e6d45dcafbb6f39650ba403c7bc38813186970d05ea686ca756716ca752"} May 20 21:48:03.875232 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:03.874914 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:05.665044 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:05.665019 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:48:05.665494 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:05.665072 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:48:05.680694 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:05.680670 2565 kubelet.go:1628] "Image garbage collection succeeded" May 20 21:48:14.881183 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:14.881153 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" May 20 21:48:14.903092 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:14.903030 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7967c8df76-6pghk" podStartSLOduration=12.229005274 podStartE2EDuration="14.903017844s" podCreationTimestamp="2026-05-20 21:48:00 +0000 UTC" firstStartedPulling="2026-05-20 21:48:01.124076371 +0000 UTC m=+295.905509601" lastFinishedPulling="2026-05-20 21:48:03.798088941 +0000 UTC m=+298.579522171" observedRunningTime="2026-05-20 21:48:03.895129404 +0000 UTC m=+298.676562668" watchObservedRunningTime="2026-05-20 21:48:14.903017844 +0000 UTC m=+309.684451095" May 20 21:48:18.984200 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:18.984165 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph"] May 20 21:48:18.986412 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:18.986389 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:18.991376 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:18.991357 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xbqdk\"" May 20 21:48:18.991521 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:18.991356 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 20 21:48:18.991601 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:18.991438 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 20 21:48:18.997589 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:18.997547 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph"] May 20 21:48:19.053514 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.053486 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntttm\" (UniqueName: \"kubernetes.io/projected/021f0089-8c88-43f2-9fdb-c04ccbf3994d-kube-api-access-ntttm\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.053660 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.053534 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/021f0089-8c88-43f2-9fdb-c04ccbf3994d-tmp\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.053660 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.053590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021f0089-8c88-43f2-9fdb-c04ccbf3994d-tls-certs\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.154063 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.154029 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/021f0089-8c88-43f2-9fdb-c04ccbf3994d-tmp\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.154222 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.154070 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021f0089-8c88-43f2-9fdb-c04ccbf3994d-tls-certs\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.154222 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.154112 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ntttm\" (UniqueName: \"kubernetes.io/projected/021f0089-8c88-43f2-9fdb-c04ccbf3994d-kube-api-access-ntttm\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.156366 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.156335 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/021f0089-8c88-43f2-9fdb-c04ccbf3994d-tmp\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.156524 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.156507 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/021f0089-8c88-43f2-9fdb-c04ccbf3994d-tls-certs\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.162607 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.162563 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntttm\" (UniqueName: \"kubernetes.io/projected/021f0089-8c88-43f2-9fdb-c04ccbf3994d-kube-api-access-ntttm\") pod \"kube-auth-proxy-7c74c79c95-mrqph\" (UID: \"021f0089-8c88-43f2-9fdb-c04ccbf3994d\") " pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.297152 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.297052 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" May 20 21:48:19.435690 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.435665 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph"] May 20 21:48:19.437328 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:48:19.437296 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod021f0089_8c88_43f2_9fdb_c04ccbf3994d.slice/crio-b888f201b7551a05eff8169f9bc603b03b469b5aa7387307c4ecfd2446090911 WatchSource:0}: Error finding container b888f201b7551a05eff8169f9bc603b03b469b5aa7387307c4ecfd2446090911: Status 404 returned error can't find the container with id b888f201b7551a05eff8169f9bc603b03b469b5aa7387307c4ecfd2446090911 May 20 21:48:19.438983 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.438967 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 21:48:19.927613 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:19.927561 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" event={"ID":"021f0089-8c88-43f2-9fdb-c04ccbf3994d","Type":"ContainerStarted","Data":"b888f201b7551a05eff8169f9bc603b03b469b5aa7387307c4ecfd2446090911"} May 20 21:48:21.829479 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.829434 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-4bbh5"] May 20 21:48:21.832133 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.832113 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:21.835031 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.834976 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 20 21:48:21.835031 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.834984 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2zf9g\"" May 20 21:48:21.841827 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.841782 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-4bbh5"] May 20 21:48:21.978444 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.978395 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-cert\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:21.978654 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:21.978504 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sd2s\" (UniqueName: \"kubernetes.io/projected/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-kube-api-access-6sd2s\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.079793 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.079707 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-cert\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.079793 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.079774 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6sd2s\" (UniqueName: \"kubernetes.io/projected/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-kube-api-access-6sd2s\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.080004 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:22.079878 2565 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 20 21:48:22.080004 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:22.079951 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-cert podName:6d22c0bd-d4fe-46e0-a08e-cf6163958c9f nodeName:}" failed. No retries permitted until 2026-05-20 21:48:22.579930673 +0000 UTC m=+317.361363910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-cert") pod "odh-model-controller-858dbf95b8-4bbh5" (UID: "6d22c0bd-d4fe-46e0-a08e-cf6163958c9f") : secret "odh-model-controller-webhook-cert" not found May 20 21:48:22.091250 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.091221 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sd2s\" (UniqueName: \"kubernetes.io/projected/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-kube-api-access-6sd2s\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.584978 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.584940 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-cert\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.587418 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.587397 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6d22c0bd-d4fe-46e0-a08e-cf6163958c9f-cert\") pod \"odh-model-controller-858dbf95b8-4bbh5\" (UID: \"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f\") " pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.745990 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.745953 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:22.939273 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.939185 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" event={"ID":"021f0089-8c88-43f2-9fdb-c04ccbf3994d","Type":"ContainerStarted","Data":"021396e800fcf94a284e051f630f50926ed71c168eace7e4e1f4f17104173d60"} May 20 21:48:22.958014 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:22.957968 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c74c79c95-mrqph" podStartSLOduration=2.164780654 podStartE2EDuration="4.957954172s" podCreationTimestamp="2026-05-20 21:48:18 +0000 UTC" firstStartedPulling="2026-05-20 21:48:19.43910954 +0000 UTC m=+314.220542770" lastFinishedPulling="2026-05-20 21:48:22.232283055 +0000 UTC m=+317.013716288" observedRunningTime="2026-05-20 21:48:22.956648269 +0000 UTC m=+317.738081520" watchObservedRunningTime="2026-05-20 21:48:22.957954172 +0000 UTC m=+317.739387421" May 20 21:48:23.082489 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:48:23.082448 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d22c0bd_d4fe_46e0_a08e_cf6163958c9f.slice/crio-51bc9519eb3334f29dedbc58a059c3e8d3a8ef84362dbd5b2f61d977728de6df WatchSource:0}: Error finding container 51bc9519eb3334f29dedbc58a059c3e8d3a8ef84362dbd5b2f61d977728de6df: Status 404 returned error can't find the container with id 51bc9519eb3334f29dedbc58a059c3e8d3a8ef84362dbd5b2f61d977728de6df May 20 21:48:23.082755 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:23.082732 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-4bbh5"] May 20 21:48:23.943566 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:23.943526 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" event={"ID":"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f","Type":"ContainerStarted","Data":"51bc9519eb3334f29dedbc58a059c3e8d3a8ef84362dbd5b2f61d977728de6df"} May 20 21:48:25.953446 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:25.953391 2565 generic.go:358] "Generic (PLEG): container finished" podID="6d22c0bd-d4fe-46e0-a08e-cf6163958c9f" containerID="2e8706d96ee7350a509629712d175106908e2fd90d55acb9aa70d4e0cd01a7a5" exitCode=1 May 20 21:48:25.954087 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:25.953487 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" event={"ID":"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f","Type":"ContainerDied","Data":"2e8706d96ee7350a509629712d175106908e2fd90d55acb9aa70d4e0cd01a7a5"} May 20 21:48:25.954087 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:25.953773 2565 scope.go:117] "RemoveContainer" containerID="2e8706d96ee7350a509629712d175106908e2fd90d55acb9aa70d4e0cd01a7a5" May 20 21:48:26.958238 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:26.958204 2565 generic.go:358] "Generic (PLEG): container finished" podID="6d22c0bd-d4fe-46e0-a08e-cf6163958c9f" containerID="d2e2ba3311629eeb7b37875bb7403cdff7fc419ad72ef4081002c8c737817c3d" exitCode=1 May 20 21:48:26.958639 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:26.958289 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" event={"ID":"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f","Type":"ContainerDied","Data":"d2e2ba3311629eeb7b37875bb7403cdff7fc419ad72ef4081002c8c737817c3d"} May 20 21:48:26.958639 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:26.958334 2565 scope.go:117] "RemoveContainer" containerID="2e8706d96ee7350a509629712d175106908e2fd90d55acb9aa70d4e0cd01a7a5" May 20 21:48:26.958639 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:26.958530 2565 scope.go:117] "RemoveContainer" containerID="d2e2ba3311629eeb7b37875bb7403cdff7fc419ad72ef4081002c8c737817c3d" May 20 21:48:26.958777 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:26.958760 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-4bbh5_opendatahub(6d22c0bd-d4fe-46e0-a08e-cf6163958c9f)\"" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" podUID="6d22c0bd-d4fe-46e0-a08e-cf6163958c9f" May 20 21:48:27.914408 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:27.914369 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-7l66d"] May 20 21:48:27.918713 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:27.918686 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:27.923055 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:27.923031 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 20 21:48:27.923055 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:27.923043 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-vqmrc\"" May 20 21:48:27.927008 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:27.926980 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-7l66d"] May 20 21:48:27.964580 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:27.964552 2565 scope.go:117] "RemoveContainer" containerID="d2e2ba3311629eeb7b37875bb7403cdff7fc419ad72ef4081002c8c737817c3d" May 20 21:48:27.964942 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:27.964812 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-4bbh5_opendatahub(6d22c0bd-d4fe-46e0-a08e-cf6163958c9f)\"" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" podUID="6d22c0bd-d4fe-46e0-a08e-cf6163958c9f" May 20 21:48:28.032405 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.032366 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba80038e-313d-42af-924a-c17b4d2dfa06-cert\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.032601 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.032438 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z9dl\" (UniqueName: \"kubernetes.io/projected/ba80038e-313d-42af-924a-c17b4d2dfa06-kube-api-access-2z9dl\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.133352 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.133314 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2z9dl\" (UniqueName: \"kubernetes.io/projected/ba80038e-313d-42af-924a-c17b4d2dfa06-kube-api-access-2z9dl\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.133532 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.133431 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba80038e-313d-42af-924a-c17b4d2dfa06-cert\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.133612 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:28.133550 2565 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 20 21:48:28.133652 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:28.133635 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba80038e-313d-42af-924a-c17b4d2dfa06-cert podName:ba80038e-313d-42af-924a-c17b4d2dfa06 nodeName:}" failed. No retries permitted until 2026-05-20 21:48:28.633617287 +0000 UTC m=+323.415050517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ba80038e-313d-42af-924a-c17b4d2dfa06-cert") pod "kserve-controller-manager-856948b99f-7l66d" (UID: "ba80038e-313d-42af-924a-c17b4d2dfa06") : secret "kserve-webhook-server-cert" not found May 20 21:48:28.142452 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.142423 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z9dl\" (UniqueName: \"kubernetes.io/projected/ba80038e-313d-42af-924a-c17b4d2dfa06-kube-api-access-2z9dl\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.637235 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.637193 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba80038e-313d-42af-924a-c17b4d2dfa06-cert\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.639609 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.639585 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ba80038e-313d-42af-924a-c17b4d2dfa06-cert\") pod \"kserve-controller-manager-856948b99f-7l66d\" (UID: \"ba80038e-313d-42af-924a-c17b4d2dfa06\") " pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.829537 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.829502 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:28.957225 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.957200 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-7l66d"] May 20 21:48:28.959237 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:48:28.959206 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba80038e_313d_42af_924a_c17b4d2dfa06.slice/crio-6168332d0937e2e0b35d39b962c126d4ad194ffa552681e92ca523ee8f17614b WatchSource:0}: Error finding container 6168332d0937e2e0b35d39b962c126d4ad194ffa552681e92ca523ee8f17614b: Status 404 returned error can't find the container with id 6168332d0937e2e0b35d39b962c126d4ad194ffa552681e92ca523ee8f17614b May 20 21:48:28.968815 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:28.968787 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" event={"ID":"ba80038e-313d-42af-924a-c17b4d2dfa06","Type":"ContainerStarted","Data":"6168332d0937e2e0b35d39b962c126d4ad194ffa552681e92ca523ee8f17614b"} May 20 21:48:32.746126 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:32.746086 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:32.746672 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:32.746651 2565 scope.go:117] "RemoveContainer" containerID="d2e2ba3311629eeb7b37875bb7403cdff7fc419ad72ef4081002c8c737817c3d" May 20 21:48:32.746918 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:48:32.746897 2565 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-4bbh5_opendatahub(6d22c0bd-d4fe-46e0-a08e-cf6163958c9f)\"" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" podUID="6d22c0bd-d4fe-46e0-a08e-cf6163958c9f" May 20 21:48:33.563586 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.563533 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf"] May 20 21:48:33.566925 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.566909 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.569759 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.569732 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 20 21:48:33.569759 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.569746 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-99tcc\"" May 20 21:48:33.569920 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.569746 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 20 21:48:33.578818 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.578793 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf"] May 20 21:48:33.684977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.684935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd7c9\" (UniqueName: \"kubernetes.io/projected/591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade-kube-api-access-gd7c9\") pod \"servicemesh-operator3-55f49c5f94-fg4jf\" (UID: \"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.684977 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.684980 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade-operator-config\") pod \"servicemesh-operator3-55f49c5f94-fg4jf\" (UID: \"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.785833 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.785797 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gd7c9\" (UniqueName: \"kubernetes.io/projected/591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade-kube-api-access-gd7c9\") pod \"servicemesh-operator3-55f49c5f94-fg4jf\" (UID: \"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.785833 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.785839 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade-operator-config\") pod \"servicemesh-operator3-55f49c5f94-fg4jf\" (UID: \"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.788339 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.788306 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade-operator-config\") pod \"servicemesh-operator3-55f49c5f94-fg4jf\" (UID: \"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.795064 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.795039 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd7c9\" (UniqueName: \"kubernetes.io/projected/591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade-kube-api-access-gd7c9\") pod \"servicemesh-operator3-55f49c5f94-fg4jf\" (UID: \"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.876974 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.876874 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:33.987481 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.987449 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" event={"ID":"ba80038e-313d-42af-924a-c17b4d2dfa06","Type":"ContainerStarted","Data":"69065cd8c65193bbb3f95de056b9ff7a74395eb9a7420bca034d915f5c5154dd"} May 20 21:48:33.987685 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:33.987539 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:48:34.038514 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:34.038460 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" podStartSLOduration=3.019329723 podStartE2EDuration="7.038438463s" podCreationTimestamp="2026-05-20 21:48:27 +0000 UTC" firstStartedPulling="2026-05-20 21:48:28.960556976 +0000 UTC m=+323.741990209" lastFinishedPulling="2026-05-20 21:48:32.979665715 +0000 UTC m=+327.761098949" observedRunningTime="2026-05-20 21:48:34.035902831 +0000 UTC m=+328.817336084" watchObservedRunningTime="2026-05-20 21:48:34.038438463 +0000 UTC m=+328.819871715" May 20 21:48:34.041763 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:34.041740 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf"] May 20 21:48:34.042508 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:48:34.042480 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod591ecbfa_e2a9_4ac2_af1f_4f2c60bf2ade.slice/crio-8bf2cfebc90ef8183e7c972e1de092230b0f7373b2cc05dfa75a5afc26056aa7 WatchSource:0}: Error finding container 8bf2cfebc90ef8183e7c972e1de092230b0f7373b2cc05dfa75a5afc26056aa7: Status 404 returned error can't find the container with id 8bf2cfebc90ef8183e7c972e1de092230b0f7373b2cc05dfa75a5afc26056aa7 May 20 21:48:34.993158 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:34.993093 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" event={"ID":"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade","Type":"ContainerStarted","Data":"8bf2cfebc90ef8183e7c972e1de092230b0f7373b2cc05dfa75a5afc26056aa7"} May 20 21:48:37.000639 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:37.000602 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" event={"ID":"591ecbfa-e2a9-4ac2-af1f-4f2c60bf2ade","Type":"ContainerStarted","Data":"5f1de476d5d203916d46fd23298421ad0d86b030129406c3d2db2fd23cfd7225"} May 20 21:48:37.001028 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:37.000730 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:37.024971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:37.024906 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" podStartSLOduration=1.5230332739999999 podStartE2EDuration="4.024886908s" podCreationTimestamp="2026-05-20 21:48:33 +0000 UTC" firstStartedPulling="2026-05-20 21:48:34.048060941 +0000 UTC m=+328.829494171" lastFinishedPulling="2026-05-20 21:48:36.549914563 +0000 UTC m=+331.331347805" observedRunningTime="2026-05-20 21:48:37.022620745 +0000 UTC m=+331.804053998" watchObservedRunningTime="2026-05-20 21:48:37.024886908 +0000 UTC m=+331.806320162" May 20 21:48:42.746943 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:42.746911 2565 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:42.747391 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:42.747233 2565 scope.go:117] "RemoveContainer" containerID="d2e2ba3311629eeb7b37875bb7403cdff7fc419ad72ef4081002c8c737817c3d" May 20 21:48:44.031719 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:44.031685 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" event={"ID":"6d22c0bd-d4fe-46e0-a08e-cf6163958c9f","Type":"ContainerStarted","Data":"db7e8e868ddcb48ad42e0374940ccb1ffd22b086882e9492b53057dbb18df37c"} May 20 21:48:44.032206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:44.031976 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:48:44.068031 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:44.067971 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" podStartSLOduration=3.08282138 podStartE2EDuration="23.067950067s" podCreationTimestamp="2026-05-20 21:48:21 +0000 UTC" firstStartedPulling="2026-05-20 21:48:23.083697661 +0000 UTC m=+317.865130890" lastFinishedPulling="2026-05-20 21:48:43.068826343 +0000 UTC m=+337.850259577" observedRunningTime="2026-05-20 21:48:44.067788574 +0000 UTC m=+338.849221826" watchObservedRunningTime="2026-05-20 21:48:44.067950067 +0000 UTC m=+338.849383320" May 20 21:48:48.006884 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:48.006847 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-fg4jf" May 20 21:48:55.037909 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:48:55.037877 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-4bbh5" May 20 21:49:04.998804 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:04.998767 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-7l66d" May 20 21:49:18.678069 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.678033 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fzn2f"] May 20 21:49:18.685652 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.685621 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:18.688864 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.688839 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 20 21:49:18.690002 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.689982 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 20 21:49:18.690118 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.690003 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-58mmf\"" May 20 21:49:18.700647 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.700623 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fzn2f"] May 20 21:49:18.828248 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.828211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pk7r\" (UniqueName: \"kubernetes.io/projected/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605-kube-api-access-7pk7r\") pod \"kuadrant-operator-catalog-fzn2f\" (UID: \"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605\") " pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:18.928882 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.928802 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7pk7r\" (UniqueName: \"kubernetes.io/projected/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605-kube-api-access-7pk7r\") pod \"kuadrant-operator-catalog-fzn2f\" (UID: \"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605\") " pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:18.937382 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.937359 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pk7r\" (UniqueName: \"kubernetes.io/projected/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605-kube-api-access-7pk7r\") pod \"kuadrant-operator-catalog-fzn2f\" (UID: \"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605\") " pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:18.996706 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:18.996661 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:19.039615 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.039583 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fzn2f"] May 20 21:49:19.144291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.144263 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fzn2f"] May 20 21:49:19.147294 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:49:19.147260 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5fbded_0c58_4c57_8b7f_7b7b0cf0d605.slice/crio-e70d58a191a3001f14a72f1dc14d1e675f8c620b41c037a9c650116e8418c279 WatchSource:0}: Error finding container e70d58a191a3001f14a72f1dc14d1e675f8c620b41c037a9c650116e8418c279: Status 404 returned error can't find the container with id e70d58a191a3001f14a72f1dc14d1e675f8c620b41c037a9c650116e8418c279 May 20 21:49:19.249291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.249214 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-drgl4"] May 20 21:49:19.252948 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.252252 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:19.258765 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.258732 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-drgl4"] May 20 21:49:19.331344 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.331316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk9fq\" (UniqueName: \"kubernetes.io/projected/1a49d832-1cb2-4f3a-9440-4f1dd208efd3-kube-api-access-pk9fq\") pod \"kuadrant-operator-catalog-drgl4\" (UID: \"1a49d832-1cb2-4f3a-9440-4f1dd208efd3\") " pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:19.432747 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.432711 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pk9fq\" (UniqueName: \"kubernetes.io/projected/1a49d832-1cb2-4f3a-9440-4f1dd208efd3-kube-api-access-pk9fq\") pod \"kuadrant-operator-catalog-drgl4\" (UID: \"1a49d832-1cb2-4f3a-9440-4f1dd208efd3\") " pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:19.441821 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.441792 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk9fq\" (UniqueName: \"kubernetes.io/projected/1a49d832-1cb2-4f3a-9440-4f1dd208efd3-kube-api-access-pk9fq\") pod \"kuadrant-operator-catalog-drgl4\" (UID: \"1a49d832-1cb2-4f3a-9440-4f1dd208efd3\") " pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:19.566468 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.566433 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:19.710497 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.710455 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-drgl4"] May 20 21:49:19.737756 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:49:19.737714 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a49d832_1cb2_4f3a_9440_4f1dd208efd3.slice/crio-bd12a0a16a79027cd6a0ed8c91e070222bcaf1e5b90d204537bc66710724af7c WatchSource:0}: Error finding container bd12a0a16a79027cd6a0ed8c91e070222bcaf1e5b90d204537bc66710724af7c: Status 404 returned error can't find the container with id bd12a0a16a79027cd6a0ed8c91e070222bcaf1e5b90d204537bc66710724af7c May 20 21:49:19.935685 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.935584 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh"] May 20 21:49:19.940455 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.940428 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:19.942981 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.942954 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 20 21:49:19.943104 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.942982 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 20 21:49:19.943104 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.943055 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4wx7d\"" May 20 21:49:19.943104 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.943077 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 20 21:49:19.943104 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.942991 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 20 21:49:19.947517 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:19.947471 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh"] May 20 21:49:20.038988 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.038955 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.039144 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.039009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.039144 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.039091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69w56\" (UniqueName: \"kubernetes.io/projected/9d80b46a-973c-43b1-ad91-d2382eb257a5-kube-api-access-69w56\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.039277 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.039175 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.039277 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.039238 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-cacerts\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.039379 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.039286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-token\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.039379 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.039337 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d80b46a-973c-43b1-ad91-d2382eb257a5-local-certs\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140092 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140049 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-token\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140092 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140095 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d80b46a-973c-43b1-ad91-d2382eb257a5-local-certs\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140117 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140139 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140185 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-69w56\" (UniqueName: \"kubernetes.io/projected/9d80b46a-973c-43b1-ad91-d2382eb257a5-kube-api-access-69w56\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140223 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140329 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140270 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-cacerts\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.140994 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.140942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.142898 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.142855 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d80b46a-973c-43b1-ad91-d2382eb257a5-local-certs\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.143125 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.143097 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.143341 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.143318 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.143420 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.143352 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d80b46a-973c-43b1-ad91-d2382eb257a5-cacerts\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.150291 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.150266 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69w56\" (UniqueName: \"kubernetes.io/projected/9d80b46a-973c-43b1-ad91-d2382eb257a5-kube-api-access-69w56\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.151710 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.151664 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d80b46a-973c-43b1-ad91-d2382eb257a5-istio-token\") pod \"istiod-openshift-gateway-798958bb55-bfkfh\" (UID: \"9d80b46a-973c-43b1-ad91-d2382eb257a5\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.152976 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.152924 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" event={"ID":"1a49d832-1cb2-4f3a-9440-4f1dd208efd3","Type":"ContainerStarted","Data":"bd12a0a16a79027cd6a0ed8c91e070222bcaf1e5b90d204537bc66710724af7c"} May 20 21:49:20.154317 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.154246 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" event={"ID":"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605","Type":"ContainerStarted","Data":"e70d58a191a3001f14a72f1dc14d1e675f8c620b41c037a9c650116e8418c279"} May 20 21:49:20.261536 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.261456 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:20.844691 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:20.844653 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh"] May 20 21:49:21.143655 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:49:21.143615 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d80b46a_973c_43b1_ad91_d2382eb257a5.slice/crio-ea411945117aa0e1850051688429115098c9283b3fc26199e77958f92a174a04 WatchSource:0}: Error finding container ea411945117aa0e1850051688429115098c9283b3fc26199e77958f92a174a04: Status 404 returned error can't find the container with id ea411945117aa0e1850051688429115098c9283b3fc26199e77958f92a174a04 May 20 21:49:21.158935 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:21.158905 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" event={"ID":"9d80b46a-973c-43b1-ad91-d2382eb257a5","Type":"ContainerStarted","Data":"ea411945117aa0e1850051688429115098c9283b3fc26199e77958f92a174a04"} May 20 21:49:22.164601 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.164544 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" event={"ID":"1a49d832-1cb2-4f3a-9440-4f1dd208efd3","Type":"ContainerStarted","Data":"c3da0396c8b0c32f71058af1f8fb06191ab68232b9a916c64f721f8f4de73751"} May 20 21:49:22.166028 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.166003 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" event={"ID":"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605","Type":"ContainerStarted","Data":"8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b"} May 20 21:49:22.166173 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.166123 2565 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" podUID="de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" containerName="registry-server" containerID="cri-o://8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b" gracePeriod=2 May 20 21:49:22.181461 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.181408 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" podStartSLOduration=1.663475076 podStartE2EDuration="3.181388983s" podCreationTimestamp="2026-05-20 21:49:19 +0000 UTC" firstStartedPulling="2026-05-20 21:49:19.739480832 +0000 UTC m=+374.520914075" lastFinishedPulling="2026-05-20 21:49:21.257394734 +0000 UTC m=+376.038827982" observedRunningTime="2026-05-20 21:49:22.178515874 +0000 UTC m=+376.959949126" watchObservedRunningTime="2026-05-20 21:49:22.181388983 +0000 UTC m=+376.962822236" May 20 21:49:22.193435 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.193391 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" podStartSLOduration=2.084944009 podStartE2EDuration="4.193374865s" podCreationTimestamp="2026-05-20 21:49:18 +0000 UTC" firstStartedPulling="2026-05-20 21:49:19.148449994 +0000 UTC m=+373.929883227" lastFinishedPulling="2026-05-20 21:49:21.256880853 +0000 UTC m=+376.038314083" observedRunningTime="2026-05-20 21:49:22.192817875 +0000 UTC m=+376.974251126" watchObservedRunningTime="2026-05-20 21:49:22.193374865 +0000 UTC m=+376.974808117" May 20 21:49:22.438617 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.438589 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:22.567408 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.567369 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pk7r\" (UniqueName: \"kubernetes.io/projected/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605-kube-api-access-7pk7r\") pod \"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605\" (UID: \"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605\") " May 20 21:49:22.570353 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.570305 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605-kube-api-access-7pk7r" (OuterVolumeSpecName: "kube-api-access-7pk7r") pod "de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" (UID: "de5fbded-0c58-4c57-8b7f-7b7b0cf0d605"). InnerVolumeSpecName "kube-api-access-7pk7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 21:49:22.669051 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:22.668959 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pk7r\" (UniqueName: \"kubernetes.io/projected/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605-kube-api-access-7pk7r\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 21:49:23.171363 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.171299 2565 generic.go:358] "Generic (PLEG): container finished" podID="de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" containerID="8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b" exitCode=0 May 20 21:49:23.171363 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.171363 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" May 20 21:49:23.171906 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.171385 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" event={"ID":"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605","Type":"ContainerDied","Data":"8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b"} May 20 21:49:23.171906 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.171433 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fzn2f" event={"ID":"de5fbded-0c58-4c57-8b7f-7b7b0cf0d605","Type":"ContainerDied","Data":"e70d58a191a3001f14a72f1dc14d1e675f8c620b41c037a9c650116e8418c279"} May 20 21:49:23.171906 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.171453 2565 scope.go:117] "RemoveContainer" containerID="8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b" May 20 21:49:23.196906 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.196877 2565 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fzn2f"] May 20 21:49:23.198714 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.198687 2565 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fzn2f"] May 20 21:49:23.629317 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.629134 2565 scope.go:117] "RemoveContainer" containerID="8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b" May 20 21:49:23.629484 ip-10-0-141-67 kubenswrapper[2565]: E0520 21:49:23.629460 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b\": container with ID starting with 8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b not found: ID does not exist" containerID="8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b" May 20 21:49:23.629551 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.629497 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b"} err="failed to get container status \"8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b\": rpc error: code = NotFound desc = could not find container \"8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b\": container with ID starting with 8e3032527dc45a8474f30510551162250ff5e7a1936386cfbe815d8f0a1ffe2b not found: ID does not exist" May 20 21:49:23.687723 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.687689 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 20 21:49:23.687826 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.687764 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 20 21:49:23.747268 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:23.747239 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" path="/var/lib/kubelet/pods/de5fbded-0c58-4c57-8b7f-7b7b0cf0d605/volumes" May 20 21:49:24.178280 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:24.178245 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" event={"ID":"9d80b46a-973c-43b1-ad91-d2382eb257a5","Type":"ContainerStarted","Data":"38b06388db62a299c1f283a3b2c7395f349758db4c97eaf06c2ce3455f855090"} May 20 21:49:24.178730 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:24.178418 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:24.180319 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:24.180293 2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-bfkfh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= May 20 21:49:24.180443 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:24.180336 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" podUID="9d80b46a-973c-43b1-ad91-d2382eb257a5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 20 21:49:24.200034 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:24.199988 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" podStartSLOduration=2.658267279 podStartE2EDuration="5.199975277s" podCreationTimestamp="2026-05-20 21:49:19 +0000 UTC" firstStartedPulling="2026-05-20 21:49:21.145760833 +0000 UTC m=+375.927194062" lastFinishedPulling="2026-05-20 21:49:23.687468831 +0000 UTC m=+378.468902060" observedRunningTime="2026-05-20 21:49:24.199808252 +0000 UTC m=+378.981241504" watchObservedRunningTime="2026-05-20 21:49:24.199975277 +0000 UTC m=+378.981408528" May 20 21:49:25.182241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:25.182208 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-bfkfh" May 20 21:49:29.567094 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:29.567054 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:29.567611 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:29.567138 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:29.588566 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:29.588541 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:30.218990 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:30.218964 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-drgl4" May 20 21:49:55.808394 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.808363 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-b94fdd4c4-4sbdf"] May 20 21:49:55.808944 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.808806 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" containerName="registry-server" May 20 21:49:55.808944 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.808824 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" containerName="registry-server" May 20 21:49:55.808944 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.808926 2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="de5fbded-0c58-4c57-8b7f-7b7b0cf0d605" containerName="registry-server" May 20 21:49:55.822507 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.822487 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.825216 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.825192 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 20 21:49:55.825540 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.825519 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b94fdd4c4-4sbdf"] May 20 21:49:55.826706 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.826515 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 20 21:49:55.826706 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.826524 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 20 21:49:55.826706 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.826548 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 20 21:49:55.826706 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.826560 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dl8fd\"" May 20 21:49:55.826973 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.826926 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 20 21:49:55.831414 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.831379 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 20 21:49:55.834941 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.834914 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-oauth-serving-cert\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.835047 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.834948 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-config\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.835047 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.834971 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-trusted-ca-bundle\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.835047 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.834992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-service-ca\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.835047 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.835023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-serving-cert\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.835270 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.835048 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msqsk\" (UniqueName: \"kubernetes.io/projected/1be37ecd-6de5-4638-a14f-e05d68a5c387-kube-api-access-msqsk\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.835270 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.835095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-oauth-config\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936231 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936196 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-config\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936231 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936236 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-trusted-ca-bundle\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936418 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936358 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-service-ca\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936460 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936413 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-serving-cert\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936498 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936464 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-msqsk\" (UniqueName: \"kubernetes.io/projected/1be37ecd-6de5-4638-a14f-e05d68a5c387-kube-api-access-msqsk\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936537 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936513 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-oauth-config\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936633 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936615 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-oauth-serving-cert\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.936955 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.936929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-config\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.937062 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.937045 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-trusted-ca-bundle\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.937129 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.937085 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-service-ca\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.937234 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.937214 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1be37ecd-6de5-4638-a14f-e05d68a5c387-oauth-serving-cert\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.938985 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.938955 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-oauth-config\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.939089 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.939021 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1be37ecd-6de5-4638-a14f-e05d68a5c387-console-serving-cert\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:55.944162 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:55.944139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msqsk\" (UniqueName: \"kubernetes.io/projected/1be37ecd-6de5-4638-a14f-e05d68a5c387-kube-api-access-msqsk\") pod \"console-b94fdd4c4-4sbdf\" (UID: \"1be37ecd-6de5-4638-a14f-e05d68a5c387\") " pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:56.134951 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:56.134867 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:49:56.274196 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:56.274159 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b94fdd4c4-4sbdf"] May 20 21:49:56.276504 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:49:56.276478 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1be37ecd_6de5_4638_a14f_e05d68a5c387.slice/crio-f15a907f4fe1ab233096c3a812d4828e492bd8f965a23776211a3ebec1705f50 WatchSource:0}: Error finding container f15a907f4fe1ab233096c3a812d4828e492bd8f965a23776211a3ebec1705f50: Status 404 returned error can't find the container with id f15a907f4fe1ab233096c3a812d4828e492bd8f965a23776211a3ebec1705f50 May 20 21:49:56.296480 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:56.296450 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b94fdd4c4-4sbdf" event={"ID":"1be37ecd-6de5-4638-a14f-e05d68a5c387","Type":"ContainerStarted","Data":"f15a907f4fe1ab233096c3a812d4828e492bd8f965a23776211a3ebec1705f50"} May 20 21:49:57.301879 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:57.301843 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b94fdd4c4-4sbdf" event={"ID":"1be37ecd-6de5-4638-a14f-e05d68a5c387","Type":"ContainerStarted","Data":"05de4313c80c5b36bb99ba12dc55f96520981f7e6494d9638a5c5e4b03108b5b"} May 20 21:49:57.320194 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:49:57.320148 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-b94fdd4c4-4sbdf" podStartSLOduration=2.32013552 podStartE2EDuration="2.32013552s" podCreationTimestamp="2026-05-20 21:49:55 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 21:49:57.318325216 +0000 UTC m=+412.099758469" watchObservedRunningTime="2026-05-20 21:49:57.32013552 +0000 UTC m=+412.101568771" May 20 21:50:06.135507 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:06.135466 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:50:06.135507 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:06.135512 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:50:06.140340 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:06.140315 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:50:06.336688 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:06.336660 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b94fdd4c4-4sbdf" May 20 21:50:46.794725 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.794690 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w"] May 20 21:50:46.799212 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.798972 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.801950 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.801926 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-q8gnd\"" May 20 21:50:46.816753 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.816725 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w"] May 20 21:50:46.882315 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882274 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882496 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882372 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fee926-105c-46ce-a688-711846bc22b1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882496 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882399 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnq7m\" (UniqueName: \"kubernetes.io/projected/25fee926-105c-46ce-a688-711846bc22b1-kube-api-access-wnq7m\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882496 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fee926-105c-46ce-a688-711846bc22b1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882496 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882477 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fee926-105c-46ce-a688-711846bc22b1-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882708 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882503 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882708 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882708 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.882708 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.882661 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983166 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fee926-105c-46ce-a688-711846bc22b1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983206 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983209 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wnq7m\" (UniqueName: \"kubernetes.io/projected/25fee926-105c-46ce-a688-711846bc22b1-kube-api-access-wnq7m\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983450 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983258 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fee926-105c-46ce-a688-711846bc22b1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983450 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983283 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fee926-105c-46ce-a688-711846bc22b1-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983450 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983311 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983450 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983337 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983450 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983378 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983450 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983428 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.983753 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983456 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.984147 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.983850 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-istio-data\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.984147 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.984014 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-workload-certs\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.984147 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.984028 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-workload-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.984147 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.984104 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-credential-socket\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.984497 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.984151 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25fee926-105c-46ce-a688-711846bc22b1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.985958 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.985936 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25fee926-105c-46ce-a688-711846bc22b1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.986110 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.986088 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25fee926-105c-46ce-a688-711846bc22b1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.991551 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.991523 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25fee926-105c-46ce-a688-711846bc22b1-istio-token\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:46.991644 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:46.991618 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnq7m\" (UniqueName: \"kubernetes.io/projected/25fee926-105c-46ce-a688-711846bc22b1-kube-api-access-wnq7m\") pod \"maas-default-gateway-openshift-default-7d6f447cf5-tl96w\" (UID: \"25fee926-105c-46ce-a688-711846bc22b1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:47.116968 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:47.116936 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:47.286186 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:47.286156 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w"] May 20 21:50:47.287855 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:50:47.287824 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25fee926_105c_46ce_a688_711846bc22b1.slice/crio-7112cb1b47dbd198cfa8c97146b5b8dd562b06922015f790fe06271fcdf7153f WatchSource:0}: Error finding container 7112cb1b47dbd198cfa8c97146b5b8dd562b06922015f790fe06271fcdf7153f: Status 404 returned error can't find the container with id 7112cb1b47dbd198cfa8c97146b5b8dd562b06922015f790fe06271fcdf7153f May 20 21:50:47.473818 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:47.473719 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" event={"ID":"25fee926-105c-46ce-a688-711846bc22b1","Type":"ContainerStarted","Data":"7112cb1b47dbd198cfa8c97146b5b8dd562b06922015f790fe06271fcdf7153f"} May 20 21:50:49.917630 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:49.917592 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 20 21:50:49.917910 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:49.917681 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 20 21:50:49.917910 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:49.917705 2565 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} May 20 21:50:50.488596 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:50.488549 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" event={"ID":"25fee926-105c-46ce-a688-711846bc22b1","Type":"ContainerStarted","Data":"62969d137aba7ed41320a623b1851ad2c501e50eb3c6fda5073071bf24904807"} May 20 21:50:50.511008 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:50.510949 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" podStartSLOduration=1.883293193 podStartE2EDuration="4.510928649s" podCreationTimestamp="2026-05-20 21:50:46 +0000 UTC" firstStartedPulling="2026-05-20 21:50:47.289699686 +0000 UTC m=+462.071132919" lastFinishedPulling="2026-05-20 21:50:49.917335132 +0000 UTC m=+464.698768375" observedRunningTime="2026-05-20 21:50:50.509528941 +0000 UTC m=+465.290962206" watchObservedRunningTime="2026-05-20 21:50:50.510928649 +0000 UTC m=+465.292361901" May 20 21:50:51.117359 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.117328 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:51.123211 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.123183 2565 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:51.233089 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.233052 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:50:51.236635 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.236612 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.239121 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.239101 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rk889\"" May 20 21:50:51.239253 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.239180 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 20 21:50:51.243979 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.243940 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:50:51.267805 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.267782 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:50:51.324537 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.324507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fngh\" (UniqueName: \"kubernetes.io/projected/3fcdb3dc-0c10-45dd-bcda-d0a019c18951-kube-api-access-7fngh\") pod \"limitador-limitador-78c99df468-5p25x\" (UID: \"3fcdb3dc-0c10-45dd-bcda-d0a019c18951\") " pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.324676 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.324557 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3fcdb3dc-0c10-45dd-bcda-d0a019c18951-config-file\") pod \"limitador-limitador-78c99df468-5p25x\" (UID: \"3fcdb3dc-0c10-45dd-bcda-d0a019c18951\") " pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.426124 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.426035 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7fngh\" (UniqueName: \"kubernetes.io/projected/3fcdb3dc-0c10-45dd-bcda-d0a019c18951-kube-api-access-7fngh\") pod \"limitador-limitador-78c99df468-5p25x\" (UID: \"3fcdb3dc-0c10-45dd-bcda-d0a019c18951\") " pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.426124 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.426105 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3fcdb3dc-0c10-45dd-bcda-d0a019c18951-config-file\") pod \"limitador-limitador-78c99df468-5p25x\" (UID: \"3fcdb3dc-0c10-45dd-bcda-d0a019c18951\") " pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.426754 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.426737 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3fcdb3dc-0c10-45dd-bcda-d0a019c18951-config-file\") pod \"limitador-limitador-78c99df468-5p25x\" (UID: \"3fcdb3dc-0c10-45dd-bcda-d0a019c18951\") " pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.434031 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.434001 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fngh\" (UniqueName: \"kubernetes.io/projected/3fcdb3dc-0c10-45dd-bcda-d0a019c18951-kube-api-access-7fngh\") pod \"limitador-limitador-78c99df468-5p25x\" (UID: \"3fcdb3dc-0c10-45dd-bcda-d0a019c18951\") " pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.492995 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.492973 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:51.494042 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.494023 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d6f447cf5-tl96w" May 20 21:50:51.547812 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.547785 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:51.684082 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:51.684059 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:50:51.685717 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:50:51.685680 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcdb3dc_0c10_45dd_bcda_d0a019c18951.slice/crio-2d239532cd62b5abfd20587c7a75d01e62f15735619ea9b81edc6a63f76964a8 WatchSource:0}: Error finding container 2d239532cd62b5abfd20587c7a75d01e62f15735619ea9b81edc6a63f76964a8: Status 404 returned error can't find the container with id 2d239532cd62b5abfd20587c7a75d01e62f15735619ea9b81edc6a63f76964a8 May 20 21:50:52.497406 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:52.497372 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" event={"ID":"3fcdb3dc-0c10-45dd-bcda-d0a019c18951","Type":"ContainerStarted","Data":"2d239532cd62b5abfd20587c7a75d01e62f15735619ea9b81edc6a63f76964a8"} May 20 21:50:54.505888 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:54.505802 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" event={"ID":"3fcdb3dc-0c10-45dd-bcda-d0a019c18951","Type":"ContainerStarted","Data":"2432fdd7bdaf782c70cf38aad930d1186a5be3c7b8bd081ff6cea2e21c0aefef"} May 20 21:50:54.505888 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:54.505868 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:50:54.523625 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:50:54.523562 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" podStartSLOduration=1.095776656 podStartE2EDuration="3.523548571s" podCreationTimestamp="2026-05-20 21:50:51 +0000 UTC" firstStartedPulling="2026-05-20 21:50:51.688398336 +0000 UTC m=+466.469831574" lastFinishedPulling="2026-05-20 21:50:54.116170244 +0000 UTC m=+468.897603489" observedRunningTime="2026-05-20 21:50:54.521411043 +0000 UTC m=+469.302844295" watchObservedRunningTime="2026-05-20 21:50:54.523548571 +0000 UTC m=+469.304981818" May 20 21:51:05.510926 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:51:05.510898 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-5p25x" May 20 21:52:11.879225 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:11.879147 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:52:26.150221 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:26.150190 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:52:33.452145 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.452106 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4"] May 20 21:52:33.455713 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.455692 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.458249 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.458227 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-d5tr2\"" May 20 21:52:33.458388 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.458307 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 20 21:52:33.459402 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.459380 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 20 21:52:33.459534 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.459455 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 20 21:52:33.464111 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.464089 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4"] May 20 21:52:33.499675 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.499641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.499833 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.499682 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2cxk\" (UniqueName: \"kubernetes.io/projected/18ab885f-127a-4588-853a-8a3507bf8a35-kube-api-access-m2cxk\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.499833 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.499751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.499833 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.499809 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.499931 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.499834 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab885f-127a-4588-853a-8a3507bf8a35-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.499931 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.499879 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.600974 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.600934 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.600974 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.600979 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m2cxk\" (UniqueName: \"kubernetes.io/projected/18ab885f-127a-4588-853a-8a3507bf8a35-kube-api-access-m2cxk\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601006 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601051 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601084 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab885f-127a-4588-853a-8a3507bf8a35-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601241 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601141 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601551 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601666 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601552 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.601666 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.601622 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.603472 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.603452 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18ab885f-127a-4588-853a-8a3507bf8a35-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.603919 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.603902 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ab885f-127a-4588-853a-8a3507bf8a35-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.609078 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.609050 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2cxk\" (UniqueName: \"kubernetes.io/projected/18ab885f-127a-4588-853a-8a3507bf8a35-kube-api-access-m2cxk\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-24kl4\" (UID: \"18ab885f-127a-4588-853a-8a3507bf8a35\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.767783 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.767694 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:33.916299 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:33.916273 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4"] May 20 21:52:33.918548 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:52:33.918513 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ab885f_127a_4588_853a_8a3507bf8a35.slice/crio-437a850e3e219dd1b57a29ab95c51ee37849ac183113b6860452a9018db00bb0 WatchSource:0}: Error finding container 437a850e3e219dd1b57a29ab95c51ee37849ac183113b6860452a9018db00bb0: Status 404 returned error can't find the container with id 437a850e3e219dd1b57a29ab95c51ee37849ac183113b6860452a9018db00bb0 May 20 21:52:34.055971 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:34.055939 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:52:34.846221 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:34.846181 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" event={"ID":"18ab885f-127a-4588-853a-8a3507bf8a35","Type":"ContainerStarted","Data":"437a850e3e219dd1b57a29ab95c51ee37849ac183113b6860452a9018db00bb0"} May 20 21:52:39.869402 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:39.869362 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" event={"ID":"18ab885f-127a-4588-853a-8a3507bf8a35","Type":"ContainerStarted","Data":"67cba3f9253621182964b5feeb7a96776e36ad6106462ffb1f652d380bd2ad7e"} May 20 21:52:40.347032 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:40.346998 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:52:44.891336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:44.891249 2565 generic.go:358] "Generic (PLEG): container finished" podID="18ab885f-127a-4588-853a-8a3507bf8a35" containerID="67cba3f9253621182964b5feeb7a96776e36ad6106462ffb1f652d380bd2ad7e" exitCode=0 May 20 21:52:44.891336 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:44.891292 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" event={"ID":"18ab885f-127a-4588-853a-8a3507bf8a35","Type":"ContainerDied","Data":"67cba3f9253621182964b5feeb7a96776e36ad6106462ffb1f652d380bd2ad7e"} May 20 21:52:45.846380 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:45.846346 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:52:46.900933 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:46.900890 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" event={"ID":"18ab885f-127a-4588-853a-8a3507bf8a35","Type":"ContainerStarted","Data":"cefeec04cac2406efae02dfeb29dc7776e7273f7230061d3c7ebb74d6716fffa"} May 20 21:52:46.901422 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:46.901170 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:52:47.533842 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.533785 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" podStartSLOduration=2.483834761 podStartE2EDuration="14.533769793s" podCreationTimestamp="2026-05-20 21:52:33 +0000 UTC" firstStartedPulling="2026-05-20 21:52:33.920186221 +0000 UTC m=+568.701619452" lastFinishedPulling="2026-05-20 21:52:45.970121254 +0000 UTC m=+580.751554484" observedRunningTime="2026-05-20 21:52:46.919032925 +0000 UTC m=+581.700466176" watchObservedRunningTime="2026-05-20 21:52:47.533769793 +0000 UTC m=+582.315203045" May 20 21:52:47.534031 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.533959 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9"] May 20 21:52:47.537686 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.537664 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.540253 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.540234 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 20 21:52:47.548437 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.548417 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9"] May 20 21:52:47.629719 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.629684 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.629874 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.629729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.629874 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.629807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.629874 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.629848 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhpg\" (UniqueName: \"kubernetes.io/projected/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-kube-api-access-xvhpg\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.629987 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.629880 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.629987 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.629928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.730958 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.730921 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731130 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.730998 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731130 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731063 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731130 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731116 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731290 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731184 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhpg\" (UniqueName: \"kubernetes.io/projected/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-kube-api-access-xvhpg\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731492 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731465 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731689 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731521 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.731812 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731690 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.732003 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.731982 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.733724 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.733697 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.733970 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.733954 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.741185 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.741161 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhpg\" (UniqueName: \"kubernetes.io/projected/4dc66efb-f5fd-476e-a4fe-aba21d11b90b-kube-api-access-xvhpg\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9\" (UID: \"4dc66efb-f5fd-476e-a4fe-aba21d11b90b\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.849086 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.849048 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:47.984984 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:47.984960 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9"] May 20 21:52:47.987231 ip-10-0-141-67 kubenswrapper[2565]: W0520 21:52:47.987205 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc66efb_f5fd_476e_a4fe_aba21d11b90b.slice/crio-a4783153a8d46dcec853c41ea676ef6a744769d2335aeb5ac648a534560c1b88 WatchSource:0}: Error finding container a4783153a8d46dcec853c41ea676ef6a744769d2335aeb5ac648a534560c1b88: Status 404 returned error can't find the container with id a4783153a8d46dcec853c41ea676ef6a744769d2335aeb5ac648a534560c1b88 May 20 21:52:48.910923 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:48.910887 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" event={"ID":"4dc66efb-f5fd-476e-a4fe-aba21d11b90b","Type":"ContainerStarted","Data":"21e8e25b9ad0e0b3afae5a4520b99a828354b7e0bca2f7d55f33bcf065342b34"} May 20 21:52:48.910923 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:48.910921 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" event={"ID":"4dc66efb-f5fd-476e-a4fe-aba21d11b90b","Type":"ContainerStarted","Data":"a4783153a8d46dcec853c41ea676ef6a744769d2335aeb5ac648a534560c1b88"} May 20 21:52:49.637267 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:49.637234 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:52:53.931315 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:53.931281 2565 generic.go:358] "Generic (PLEG): container finished" podID="4dc66efb-f5fd-476e-a4fe-aba21d11b90b" containerID="21e8e25b9ad0e0b3afae5a4520b99a828354b7e0bca2f7d55f33bcf065342b34" exitCode=0 May 20 21:52:53.931651 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:53.931347 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" event={"ID":"4dc66efb-f5fd-476e-a4fe-aba21d11b90b","Type":"ContainerDied","Data":"21e8e25b9ad0e0b3afae5a4520b99a828354b7e0bca2f7d55f33bcf065342b34"} May 20 21:52:54.936787 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:54.936751 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" event={"ID":"4dc66efb-f5fd-476e-a4fe-aba21d11b90b","Type":"ContainerStarted","Data":"676d7ec080da1bc319eeff71f8b4000ccb6304f0039db74fc7b50568872cf1b1"} May 20 21:52:54.937189 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:54.936974 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:52:54.954775 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:54.954728 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" podStartSLOduration=7.747759439 podStartE2EDuration="7.954715627s" podCreationTimestamp="2026-05-20 21:52:47 +0000 UTC" firstStartedPulling="2026-05-20 21:52:53.931988293 +0000 UTC m=+588.713421523" lastFinishedPulling="2026-05-20 21:52:54.138944481 +0000 UTC m=+588.920377711" observedRunningTime="2026-05-20 21:52:54.954031799 +0000 UTC m=+589.735465077" watchObservedRunningTime="2026-05-20 21:52:54.954715627 +0000 UTC m=+589.736148878" May 20 21:52:57.918774 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:52:57.918744 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-24kl4" May 20 21:53:05.700916 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:53:05.700886 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:53:05.701781 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:53:05.701759 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:53:05.953733 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:53:05.953647 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9" May 20 21:53:49.639239 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:53:49.639205 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:53:59.537002 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:53:59.536964 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:54:08.437516 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:54:08.437477 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:54:19.038612 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:54:19.038558 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:54:26.945111 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:54:26.945071 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:54:37.854224 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:54:37.854190 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:55:10.937954 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:55:10.937871 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:55:15.186598 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:55:15.186540 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:55:50.840506 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:55:50.840467 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:56:06.236144 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:56:06.236110 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:56:44.545549 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:56:44.545508 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:57:01.041169 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:57:01.041135 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:57:15.351323 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:57:15.351287 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:57:32.153848 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:57:32.153806 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:57:57.486813 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:57:57.486733 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:58:01.794662 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:01.794629 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:58:05.725511 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:05.725480 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:58:05.728076 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:05.728055 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 21:58:24.042760 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:24.042728 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:58:33.670800 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:33.670760 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:58:49.747826 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:49.747796 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:58:58.655047 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:58:58.655009 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:59:15.040937 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:59:15.040905 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:59:23.055552 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:59:23.055517 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 21:59:55.841436 ip-10-0-141-67 kubenswrapper[2565]: I0520 21:59:55.841402 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:00.141771 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.141734 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29655240-5wmcg"] May 20 22:00:00.145209 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.145185 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:00.147686 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.147663 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-xr2b8\"" May 20 22:00:00.158096 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.158074 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29655240-5wmcg"] May 20 22:00:00.184359 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.184333 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cz8t\" (UniqueName: \"kubernetes.io/projected/b76c7ee1-976d-4769-b543-160c0ff45589-kube-api-access-6cz8t\") pod \"maas-api-key-cleanup-29655240-5wmcg\" (UID: \"b76c7ee1-976d-4769-b543-160c0ff45589\") " pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:00.284789 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.284758 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6cz8t\" (UniqueName: \"kubernetes.io/projected/b76c7ee1-976d-4769-b543-160c0ff45589-kube-api-access-6cz8t\") pod \"maas-api-key-cleanup-29655240-5wmcg\" (UID: \"b76c7ee1-976d-4769-b543-160c0ff45589\") " pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:00.293540 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.293513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cz8t\" (UniqueName: \"kubernetes.io/projected/b76c7ee1-976d-4769-b543-160c0ff45589-kube-api-access-6cz8t\") pod \"maas-api-key-cleanup-29655240-5wmcg\" (UID: \"b76c7ee1-976d-4769-b543-160c0ff45589\") " pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:00.455610 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.455510 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:00.586432 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.586405 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29655240-5wmcg"] May 20 22:00:00.588638 ip-10-0-141-67 kubenswrapper[2565]: W0520 22:00:00.588613 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76c7ee1_976d_4769_b543_160c0ff45589.slice/crio-1f3c352f43d4360acfa6e8ec110d326d21578c56b5b7aaa912364a2483c2b751 WatchSource:0}: Error finding container 1f3c352f43d4360acfa6e8ec110d326d21578c56b5b7aaa912364a2483c2b751: Status 404 returned error can't find the container with id 1f3c352f43d4360acfa6e8ec110d326d21578c56b5b7aaa912364a2483c2b751 May 20 22:00:00.590734 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:00.590717 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 22:00:01.469005 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:01.468966 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" event={"ID":"b76c7ee1-976d-4769-b543-160c0ff45589","Type":"ContainerStarted","Data":"1f3c352f43d4360acfa6e8ec110d326d21578c56b5b7aaa912364a2483c2b751"} May 20 22:00:04.340519 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:04.340482 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:12.510220 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:12.510128 2565 generic.go:358] "Generic (PLEG): container finished" podID="b76c7ee1-976d-4769-b543-160c0ff45589" containerID="43629abecaf755af455c48e4cc9959edbf9c2d6574da7194fa1326886f9ca8ce" exitCode=0 May 20 22:00:12.510557 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:12.510213 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" event={"ID":"b76c7ee1-976d-4769-b543-160c0ff45589","Type":"ContainerDied","Data":"43629abecaf755af455c48e4cc9959edbf9c2d6574da7194fa1326886f9ca8ce"} May 20 22:00:13.638454 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:13.638424 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:13.645266 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:13.645245 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:13.705553 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:13.705522 2565 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cz8t\" (UniqueName: \"kubernetes.io/projected/b76c7ee1-976d-4769-b543-160c0ff45589-kube-api-access-6cz8t\") pod \"b76c7ee1-976d-4769-b543-160c0ff45589\" (UID: \"b76c7ee1-976d-4769-b543-160c0ff45589\") " May 20 22:00:13.707864 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:13.707838 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76c7ee1-976d-4769-b543-160c0ff45589-kube-api-access-6cz8t" (OuterVolumeSpecName: "kube-api-access-6cz8t") pod "b76c7ee1-976d-4769-b543-160c0ff45589" (UID: "b76c7ee1-976d-4769-b543-160c0ff45589"). InnerVolumeSpecName "kube-api-access-6cz8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 20 22:00:13.806805 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:13.806782 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cz8t\" (UniqueName: \"kubernetes.io/projected/b76c7ee1-976d-4769-b543-160c0ff45589-kube-api-access-6cz8t\") on node \"ip-10-0-141-67.ec2.internal\" DevicePath \"\"" May 20 22:00:14.518428 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:14.518397 2565 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" May 20 22:00:14.518624 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:14.518395 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29655240-5wmcg" event={"ID":"b76c7ee1-976d-4769-b543-160c0ff45589","Type":"ContainerDied","Data":"1f3c352f43d4360acfa6e8ec110d326d21578c56b5b7aaa912364a2483c2b751"} May 20 22:00:14.518624 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:14.518502 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f3c352f43d4360acfa6e8ec110d326d21578c56b5b7aaa912364a2483c2b751" May 20 22:00:20.951356 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:20.951323 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:29.238451 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:29.238419 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:37.639052 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:37.639015 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:41.940884 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:41.940845 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:46.341419 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:46.341380 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:00:57.445388 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:00:57.445295 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:01:43.640007 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:01:43.639977 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:01:49.347945 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:01:49.347911 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:01:55.712555 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:01:55.712518 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:04.532485 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:04.532448 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:13.946310 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:13.946279 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:21.794358 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:21.794319 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:31.146208 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:31.146127 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:38.746486 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:38.746449 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:47.841451 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:47.841417 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:02:56.138521 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:02:56.138489 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:05.752165 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:05.752114 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:03:05.756278 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:05.756240 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:03:06.437305 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:06.437266 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:14.250100 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:14.250068 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:22.841186 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:22.841149 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:31.244670 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:31.244640 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:40.239050 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:40.239018 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:48.142269 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:48.142236 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:03:58.156402 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:03:58.156317 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:04:06.336522 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:04:06.336479 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:06:22.346553 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:06:22.346519 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:06:27.643197 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:06:27.643158 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:06:53.746194 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:06:53.746161 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:00.587309 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:00.587228 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:09.663926 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:09.663890 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:20.841313 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:20.841280 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:28.946975 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:28.946934 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:39.640337 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:39.640297 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:48.345123 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:48.345085 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:07:59.040349 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:07:59.040311 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:08:05.777083 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:08:05.777045 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:08:05.784334 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:08:05.784302 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:08:08.635128 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:08:08.635090 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:08:18.551538 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:08:18.551504 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:08:27.437624 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:08:27.437546 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:09:00.547933 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:09:00.547901 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:09:43.547510 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:09:43.547467 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:09:51.412513 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:09:51.412481 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:00.740222 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:00.740139 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:09.252335 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:09.252292 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:18.245394 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:18.245357 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:28.561270 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:28.561231 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:37.253763 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:37.253731 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:45.171365 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:45.171321 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:10:53.012794 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:10:53.012762 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:11:01.338377 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:11:01.338344 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:11:10.144519 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:11:10.144476 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:11:20.541226 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:11:20.541192 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:11:37.847982 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:11:37.847946 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:11:47.049118 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:11:47.049085 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:11:55.044637 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:11:55.044600 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:12:03.143085 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:12:03.143050 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:12:20.855439 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:12:20.855403 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:12:28.649613 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:12:28.649564 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:12:37.545951 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:12:37.545911 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:12:46.741463 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:12:46.741425 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:12:56.145964 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:12:56.145892 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:13:03.940707 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:03.940674 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:13:05.802463 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:05.802435 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:13:05.809966 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:05.809947 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:13:13.046412 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:13.046372 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:13:25.740804 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:25.740771 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:13:34.348734 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:34.348698 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:13:45.843246 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:45.843209 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:13:55.352065 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:13:55.352025 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:14:01.332921 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:14:01.332884 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:14:11.941669 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:14:11.941638 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:14:18.137374 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:14:18.137339 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:14:35.757454 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:14:35.757420 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:14:45.049892 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:14:45.049854 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:14:53.347613 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:14:53.347562 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:15:00.651517 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:00.651485 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:15:25.047218 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:25.047183 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:15:37.546119 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:37.546083 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-5p25x"] May 20 22:15:43.517926 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:43.517894 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-7l66d_ba80038e-313d-42af-924a-c17b4d2dfa06/manager/0.log" May 20 22:15:43.752547 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:43.752519 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29655240-5wmcg_b76c7ee1-976d-4769-b543-160c0ff45589/cleanup/0.log" May 20 22:15:44.088071 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:44.088040 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-4bbh5_6d22c0bd-d4fe-46e0-a08e-cf6163958c9f/manager/2.log" May 20 22:15:44.207591 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:44.207539 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7967c8df76-6pghk_fa465369-893a-4dce-8da9-1029d174b580/manager/0.log" May 20 22:15:46.161614 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:46.161564 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-drgl4_1a49d832-1cb2-4f3a-9440-4f1dd208efd3/registry-server/0.log" May 20 22:15:46.411748 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:46.411664 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-5p25x_3fcdb3dc-0c10-45dd-bcda-d0a019c18951/limitador/0.log" May 20 22:15:46.970275 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:46.970240 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-bfkfh_9d80b46a-973c-43b1-ad91-d2382eb257a5/discovery/0.log" May 20 22:15:47.085931 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:47.085903 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c74c79c95-mrqph_021f0089-8c88-43f2-9fdb-c04ccbf3994d/kube-auth-proxy/0.log" May 20 22:15:47.302624 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:47.302596 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-tl96w_25fee926-105c-46ce-a688-711846bc22b1/istio-proxy/0.log" May 20 22:15:47.513588 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:47.513548 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58f97b448b-g9d6k_150654bb-1c66-4f64-97dd-5a1cdd4d370f/router/0.log" May 20 22:15:48.050522 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:48.050496 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-24kl4_18ab885f-127a-4588-853a-8a3507bf8a35/storage-initializer/0.log" May 20 22:15:48.058477 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:48.058450 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-24kl4_18ab885f-127a-4588-853a-8a3507bf8a35/main/0.log" May 20 22:15:48.274829 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:48.274793 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9_4dc66efb-f5fd-476e-a4fe-aba21d11b90b/main/0.log" May 20 22:15:48.281700 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:48.281677 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-68nd9_4dc66efb-f5fd-476e-a4fe-aba21d11b90b/storage-initializer/0.log" May 20 22:15:54.806444 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:54.806415 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-txkmq_7e070039-fa4d-45ae-8c0a-7c5abaa23e35/global-pull-secret-syncer/0.log" May 20 22:15:54.927755 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:54.927724 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xzvgm_3ae77780-68cf-466d-80a6-8c7fd5053fad/konnectivity-agent/0.log" May 20 22:15:54.997792 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:54.997745 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-67.ec2.internal_ec8abf2a5fcc8cf40a4e9747a451ea23/haproxy/0.log" May 20 22:15:59.557474 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:59.557439 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-drgl4_1a49d832-1cb2-4f3a-9440-4f1dd208efd3/registry-server/0.log" May 20 22:15:59.651177 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:15:59.651149 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-5p25x_3fcdb3dc-0c10-45dd-bcda-d0a019c18951/limitador/0.log" May 20 22:16:01.406665 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:01.406635 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-72ztf_7eb15ff5-c1cf-40e2-a612-7e5494cc395e/cluster-monitoring-operator/0.log" May 20 22:16:01.800110 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:01.800031 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vxm65_646bd61c-fc09-4da3-b804-f6947b5e451f/node-exporter/0.log" May 20 22:16:01.824537 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:01.824503 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vxm65_646bd61c-fc09-4da3-b804-f6947b5e451f/kube-rbac-proxy/0.log" May 20 22:16:01.845447 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:01.845425 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vxm65_646bd61c-fc09-4da3-b804-f6947b5e451f/init-textfile/0.log" May 20 22:16:02.179746 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:02.179715 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6c4bbc67b7-lsrwr_4df9c042-0aaf-4533-ab9e-7c72222a641f/prometheus-operator-admission-webhook/0.log" May 20 22:16:03.446441 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:03.446404 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-9gf48_09b71b40-b58b-4831-ac46-0f4c09f6f287/networking-console-plugin/0.log" May 20 22:16:03.998557 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:03.998526 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/1.log" May 20 22:16:04.007665 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.007628 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-44f4f_291a4427-81e3-47f4-b3cf-26ce3d5911b7/console-operator/2.log" May 20 22:16:04.413152 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.413118 2565 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg"] May 20 22:16:04.413500 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.413486 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76c7ee1-976d-4769-b543-160c0ff45589" containerName="cleanup" May 20 22:16:04.413500 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.413500 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76c7ee1-976d-4769-b543-160c0ff45589" containerName="cleanup" May 20 22:16:04.413665 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.413583 2565 memory_manager.go:356] "RemoveStaleState removing state" podUID="b76c7ee1-976d-4769-b543-160c0ff45589" containerName="cleanup" May 20 22:16:04.416985 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.416965 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.419704 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.419680 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q688f\"/\"openshift-service-ca.crt\"" May 20 22:16:04.419832 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.419812 2565 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q688f\"/\"kube-root-ca.crt\"" May 20 22:16:04.420931 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.420911 2565 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q688f\"/\"default-dockercfg-krzjx\"" May 20 22:16:04.430002 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.429979 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg"] May 20 22:16:04.472370 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.472339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-proc\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.472740 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.472377 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5bfp\" (UniqueName: \"kubernetes.io/projected/1ee809ef-d821-4fea-be1c-2f9c9180fc25-kube-api-access-p5bfp\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.472740 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.472403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-podres\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.472740 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.472482 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-sys\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.472740 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.472511 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-lib-modules\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.540854 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.540824 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-b94fdd4c4-4sbdf_1be37ecd-6de5-4638-a14f-e05d68a5c387/console/0.log" May 20 22:16:04.573185 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573150 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-proc\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573421 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573197 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p5bfp\" (UniqueName: \"kubernetes.io/projected/1ee809ef-d821-4fea-be1c-2f9c9180fc25-kube-api-access-p5bfp\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573421 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573235 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-podres\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573421 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573268 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-sys\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573421 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573268 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-proc\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573421 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573292 2565 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-lib-modules\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573421 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573335 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-sys\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573745 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573428 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-podres\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.573745 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.573448 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ee809ef-d821-4fea-be1c-2f9c9180fc25-lib-modules\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.578379 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.578360 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-2nbqz_a584980a-67ce-4416-b2d0-180c64eafacd/download-server/0.log" May 20 22:16:04.581980 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.581957 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5bfp\" (UniqueName: \"kubernetes.io/projected/1ee809ef-d821-4fea-be1c-2f9c9180fc25-kube-api-access-p5bfp\") pod \"perf-node-gather-daemonset-f49sg\" (UID: \"1ee809ef-d821-4fea-be1c-2f9c9180fc25\") " pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.727593 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.727477 2565 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:04.860537 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.860513 2565 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg"] May 20 22:16:04.862114 ip-10-0-141-67 kubenswrapper[2565]: W0520 22:16:04.862084 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1ee809ef_d821_4fea_be1c_2f9c9180fc25.slice/crio-7383bc21efedc937fbd73a9580bca4540748bdbf6aa4c11933480f42330153fb WatchSource:0}: Error finding container 7383bc21efedc937fbd73a9580bca4540748bdbf6aa4c11933480f42330153fb: Status 404 returned error can't find the container with id 7383bc21efedc937fbd73a9580bca4540748bdbf6aa4c11933480f42330153fb May 20 22:16:04.863628 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.863612 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 20 22:16:04.957251 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:04.957213 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" event={"ID":"1ee809ef-d821-4fea-be1c-2f9c9180fc25","Type":"ContainerStarted","Data":"7383bc21efedc937fbd73a9580bca4540748bdbf6aa4c11933480f42330153fb"} May 20 22:16:05.099799 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.099769 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-wcwl5_cad1b37a-2384-40f9-aba0-8d8f1f9ee0ad/volume-data-source-validator/0.log" May 20 22:16:05.849497 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.849465 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xhk7_d1eddfa4-fc63-448b-b6f9-2e321d8b02f2/dns/0.log" May 20 22:16:05.875929 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.875901 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6xhk7_d1eddfa4-fc63-448b-b6f9-2e321d8b02f2/kube-rbac-proxy/0.log" May 20 22:16:05.962672 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.962639 2565 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" event={"ID":"1ee809ef-d821-4fea-be1c-2f9c9180fc25","Type":"ContainerStarted","Data":"84b1071aa6ff6d81ff83848d5ab584a5b92a344040ebdbfe641b279c65ba6315"} May 20 22:16:05.962853 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.962787 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:05.981850 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.981796 2565 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" podStartSLOduration=1.981778488 podStartE2EDuration="1.981778488s" podCreationTimestamp="2026-05-20 22:16:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-20 22:16:05.979711648 +0000 UTC m=+1980.761144902" watchObservedRunningTime="2026-05-20 22:16:05.981778488 +0000 UTC m=+1980.763211741" May 20 22:16:05.995796 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:05.995753 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-47h4c_c5e8afa1-caeb-4f0c-9d11-2ec5cea5a0d2/dns-node-resolver/0.log" May 20 22:16:06.573402 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:06.573369 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-knkhq_64b82855-f1bb-4887-8207-26154822ff4e/node-ca/0.log" May 20 22:16:07.580366 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:07.580337 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-bfkfh_9d80b46a-973c-43b1-ad91-d2382eb257a5/discovery/0.log" May 20 22:16:07.602782 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:07.602738 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c74c79c95-mrqph_021f0089-8c88-43f2-9fdb-c04ccbf3994d/kube-auth-proxy/0.log" May 20 22:16:07.709922 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:07.709886 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d6f447cf5-tl96w_25fee926-105c-46ce-a688-711846bc22b1/istio-proxy/0.log" May 20 22:16:07.793422 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:07.793393 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58f97b448b-g9d6k_150654bb-1c66-4f64-97dd-5a1cdd4d370f/router/0.log" May 20 22:16:08.359685 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:08.359655 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6hfxx_7b5d97ad-7543-4fd6-aabb-7e6a4e1c9e7b/serve-healthcheck-canary/0.log" May 20 22:16:08.900529 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:08.900491 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-kxsw4_55be9913-a333-4eef-90a1-c2b3311cf9a6/insights-operator/0.log" May 20 22:16:08.901430 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:08.901400 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-kxsw4_55be9913-a333-4eef-90a1-c2b3311cf9a6/insights-operator/1.log" May 20 22:16:08.925388 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:08.925359 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9v4xs_a88b42d3-8f3d-4847-861e-f33c7c04274d/kube-rbac-proxy/0.log" May 20 22:16:08.949821 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:08.949798 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9v4xs_a88b42d3-8f3d-4847-861e-f33c7c04274d/exporter/0.log" May 20 22:16:08.972112 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:08.972090 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9v4xs_a88b42d3-8f3d-4847-861e-f33c7c04274d/extractor/0.log" May 20 22:16:11.178496 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:11.178466 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-7l66d_ba80038e-313d-42af-924a-c17b4d2dfa06/manager/0.log" May 20 22:16:11.242713 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:11.242687 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29655240-5wmcg_b76c7ee1-976d-4769-b543-160c0ff45589/cleanup/0.log" May 20 22:16:11.402371 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:11.402336 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-4bbh5_6d22c0bd-d4fe-46e0-a08e-cf6163958c9f/manager/1.log" May 20 22:16:11.424659 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:11.424632 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-4bbh5_6d22c0bd-d4fe-46e0-a08e-cf6163958c9f/manager/2.log" May 20 22:16:11.477214 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:11.477126 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7967c8df76-6pghk_fa465369-893a-4dce-8da9-1029d174b580/manager/0.log" May 20 22:16:11.975767 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:11.975740 2565 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q688f/perf-node-gather-daemonset-f49sg" May 20 22:16:17.254044 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:17.254012 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-qrwmw_a20fd981-b049-45dc-bf1f-1180f4205203/migrator/0.log" May 20 22:16:17.273231 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:17.273207 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-qrwmw_a20fd981-b049-45dc-bf1f-1180f4205203/graceful-termination/0.log" May 20 22:16:17.625369 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:17.625334 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-tlvqp_46a4c312-92f4-4a59-8acd-96a2c204d00c/kube-storage-version-migrator-operator/1.log" May 20 22:16:17.626853 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:17.626826 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-tlvqp_46a4c312-92f4-4a59-8acd-96a2c204d00c/kube-storage-version-migrator-operator/0.log" May 20 22:16:18.659845 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:18.659812 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9bh8z_325e3bc6-4b21-490f-b8d4-8344c0e5b6c7/kube-multus/0.log" May 20 22:16:19.023830 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.023762 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/kube-multus-additional-cni-plugins/0.log" May 20 22:16:19.043815 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.043789 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/egress-router-binary-copy/0.log" May 20 22:16:19.062691 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.062664 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/cni-plugins/0.log" May 20 22:16:19.082230 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.082201 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/bond-cni-plugin/0.log" May 20 22:16:19.104179 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.104153 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/routeoverride-cni/0.log" May 20 22:16:19.124254 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.124236 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/whereabouts-cni-bincopy/0.log" May 20 22:16:19.143873 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.143852 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d2q8g_af93b9bc-b94e-43f9-a980-11e0a7f3d6f6/whereabouts-cni/0.log" May 20 22:16:19.224268 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.224242 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2slgv_a5df8c31-3ae1-4ae3-9a87-3da35016fd6f/network-metrics-daemon/0.log" May 20 22:16:19.240925 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:19.240903 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2slgv_a5df8c31-3ae1-4ae3-9a87-3da35016fd6f/kube-rbac-proxy/0.log" May 20 22:16:20.566755 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.566727 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/ovn-controller/0.log" May 20 22:16:20.601927 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.601907 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/ovn-acl-logging/0.log" May 20 22:16:20.625671 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.625648 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/kube-rbac-proxy-node/0.log" May 20 22:16:20.645220 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.645197 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/kube-rbac-proxy-ovn-metrics/0.log" May 20 22:16:20.663918 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.663895 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/northd/0.log" May 20 22:16:20.684188 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.684167 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/nbdb/0.log" May 20 22:16:20.705894 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.705875 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/sbdb/0.log" May 20 22:16:20.859081 ip-10-0-141-67 kubenswrapper[2565]: I0520 22:16:20.859016 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-l6m8q_716ecc43-ddb8-4408-988f-17cf83c4e8a5/ovnkube-controller/0.log"