May 28 17:06:50.228946 ip-10-0-141-155 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 28 17:06:50.228959 ip-10-0-141-155 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 28 17:06:50.228968 ip-10-0-141-155 systemd[1]: kubelet.service: Failed with result 'resources'.
May 28 17:06:50.229277 ip-10-0-141-155 systemd[1]: Failed to start Kubernetes Kubelet.
May 28 17:07:00.402149 ip-10-0-141-155 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 28 17:07:00.402169 ip-10-0-141-155 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c08a1cabede744f8b28a569954e18ca7 --
May 28 17:09:39.235617 ip-10-0-141-155 systemd[1]: Starting Kubernetes Kubelet...
May 28 17:09:39.731705 ip-10-0-141-155 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:09:39.731705 ip-10-0-141-155 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 17:09:39.731705 ip-10-0-141-155 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:09:39.731705 ip-10-0-141-155 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 17:09:39.731705 ip-10-0-141-155 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 17:09:39.734640 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.734547    2574 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 17:09:39.741983 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741960    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:09:39.741983 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741977    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:09:39.741983 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741981    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:09:39.741983 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741984    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:09:39.741983 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741987    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:09:39.741983 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741991    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741993    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.741997    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742000    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742002    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742006    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742009    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742011    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742014    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742017    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742020    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742023    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742025    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742028    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742030    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742033    2574 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742036    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742038    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742045    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742048    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:09:39.742202 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742051    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742053    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742056    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742058    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742061    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742065    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742070    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742074    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742077    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742080    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742084    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742086    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742090    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742092    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742097    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742100    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742103    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742106    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:09:39.742715 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742109    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742111    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742114    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742117    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742119    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742122    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742125    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742127    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742130    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742133    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742135    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742138    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742140    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742142    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742145    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742171    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742174    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742177    2574 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742180    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742182    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:09:39.743154 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742185    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742188    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742190    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742193    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742196    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742199    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742201    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742204    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742208    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742211    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742214    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742218    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742233    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742244    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742248    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742253    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742256    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742259    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742262    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742265    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:09:39.743655 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742267    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742271    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742274    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742663    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742668    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742672    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742677    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742680    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742683    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742686    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742689    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742692    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742695    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742697    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742700    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742702    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742705    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742708    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742710    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:09:39.744140 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742713    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742716    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742719    2574 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742722    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742725    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742728    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742730    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742733    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742736    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742738    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742741    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742743    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742746    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742748    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742751    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742753    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742756    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742758    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742761    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742763    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:09:39.744612 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742766    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742768    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742771    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742775    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742779    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742781    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742784    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742787    2574 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742789    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742792    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742794    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742797    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742800    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742803    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742806    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742808    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742811    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742814    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742816    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:09:39.745147 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742819    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742821    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742824    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742827    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742829    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742832    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742834    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742836    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742839    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742841    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742844    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742846    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742849    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742851    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742854    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742857    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742860    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742862    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742865    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742868    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:09:39.745624 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742870    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742873    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742875    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742878    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742880    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742883    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742885    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742887    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742891    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742894    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.742896    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743778    2574 flags.go:64] FLAG: --address="0.0.0.0"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743790    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743795    2574 flags.go:64] FLAG: --anonymous-auth="true"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743800    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743806    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743809    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743813    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743818    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743821    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743824    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 17:09:39.746106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743828    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743832    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743835    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743838    2574 flags.go:64] FLAG: --cgroup-root=""
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743840    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743843    2574 flags.go:64] FLAG: --client-ca-file=""
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743847    2574 flags.go:64] FLAG: --cloud-config=""
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743850    2574 flags.go:64] FLAG: --cloud-provider="external"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743853    2574 flags.go:64] FLAG: --cluster-dns="[]"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743857    2574 flags.go:64] FLAG: --cluster-domain=""
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743860    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743863    2574 flags.go:64] FLAG: --config-dir=""
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743866    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743869    2574 flags.go:64] FLAG: --container-log-max-files="5"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743873    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743876    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743879    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743882    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743885    2574 flags.go:64] FLAG: --contention-profiling="false"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743888    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743891    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743894    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743897    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743901    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743904    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 17:09:39.746632 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743907    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743910    2574 flags.go:64] FLAG: --enable-load-reader="false"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743914    2574 flags.go:64] FLAG: --enable-server="true"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743917    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743920    2574 flags.go:64] FLAG: --event-burst="100"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743924    2574 flags.go:64] FLAG: --event-qps="50"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743926    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743930    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743933    2574 flags.go:64] FLAG: --eviction-hard=""
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743937    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743940    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743943    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743946    2574 flags.go:64] FLAG: --eviction-soft=""
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743949    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743952    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743955    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743958    2574 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743961    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743964    2574 flags.go:64] FLAG: --fail-swap-on="true"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743967    2574 flags.go:64] FLAG: --feature-gates=""
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743971    2574 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743974    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743977    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743981    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743984    2574 flags.go:64] FLAG: --healthz-port="10248"
May 28 17:09:39.747213 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743987    2574 flags.go:64] FLAG: --help="false"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743989    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-141-155.ec2.internal"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743993    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743996    2574 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.743999    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744002    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744005    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744008    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744011    2574 flags.go:64] FLAG: --image-service-endpoint=""
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744014    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744018    2574 flags.go:64] FLAG: --kube-api-burst="100"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744021    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744024    2574 flags.go:64] FLAG: --kube-api-qps="50"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744027    2574 flags.go:64] FLAG: --kube-reserved=""
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744030    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744033    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744036    2574 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744039    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744042    2574 flags.go:64] FLAG: --lock-file=""
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744045    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744048    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744051    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744057    2574 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 17:09:39.747825 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744060    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744063    2574 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744066    2574 flags.go:64] FLAG: --logging-format="text"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744069    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744072    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744076    2574 flags.go:64] FLAG: --manifest-url=""
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744078    2574 flags.go:64] FLAG: --manifest-url-header=""
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744083    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744086    2574 flags.go:64] FLAG: --max-open-files="1000000"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744090    2574 flags.go:64] FLAG: --max-pods="110"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744093    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744096    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744099    2574 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744102    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744105    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744109    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744112    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744119    2574 flags.go:64] FLAG: --node-status-max-images="50"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744122    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744125    2574 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744129    2574 flags.go:64] FLAG: --pod-cidr=""
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744132    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744137    2574 flags.go:64] FLAG: --pod-manifest-path=""
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744140    2574 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 17:09:39.748403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744144    2574 flags.go:64] FLAG: --pods-per-core="0"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744147    2574 flags.go:64] FLAG: --port="10250"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744150    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744153    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a820d0978a5e6fe1"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744156    2574 flags.go:64] FLAG: --qos-reserved=""
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744160    2574 flags.go:64] FLAG: --read-only-port="10255"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744163    2574 flags.go:64] FLAG: --register-node="true"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744165    2574 flags.go:64] FLAG: --register-schedulable="true"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744168    2574 flags.go:64] FLAG: --register-with-taints=""
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744172    2574 flags.go:64] FLAG: --registry-burst="10"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744175    2574 flags.go:64] FLAG: --registry-qps="5"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744177    2574 flags.go:64] FLAG: --reserved-cpus=""
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744180    2574 flags.go:64] FLAG: --reserved-memory=""
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744184    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744187    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744190    2574 flags.go:64] FLAG: --rotate-certificates="false"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744192    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744195    2574 flags.go:64] FLAG: --runonce="false"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744198    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744201    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744204    2574 flags.go:64] FLAG: --seccomp-default="false"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744206    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744209    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744212    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744215    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744218    2574 flags.go:64] FLAG: --storage-driver-password="root"
May 28 17:09:39.748973 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744221    2574 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744224    2574 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744226    2574 flags.go:64] FLAG: --storage-driver-user="root"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744230    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744233    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744236    2574 flags.go:64] FLAG: --system-cgroups=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744241    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744246    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744249    2574 flags.go:64] FLAG: --tls-cert-file=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744252    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744256    2574 flags.go:64] FLAG: --tls-min-version=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744259    2574 flags.go:64] FLAG: --tls-private-key-file=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744264    2574 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744268    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744270    2574 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744274    2574 flags.go:64] FLAG: --v="2"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744278    2574 flags.go:64] FLAG: --version="false"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744282    2574 flags.go:64] FLAG: --vmodule=""
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744287    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.744290    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744399    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744403    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744406    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744409    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:09:39.749598 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744412    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744415    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744418    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744421    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744423    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744426    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744429    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744432    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744434    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744437    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744439    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744442    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744451    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744453    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744458    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744460    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744463    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744466    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744468    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744471    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:09:39.750209 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744475    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744477    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744481    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744483    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744486    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744488    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744491    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744494    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744496    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744499    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744502    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744504    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744507    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744509    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744512    2574 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744514    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744517    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744520    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744522    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744525    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:09:39.750740 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744527    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744530    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744532    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744535    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744539    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744543    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744548    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744551    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744554    2574 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744557    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744559    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744562    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744566    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744569    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744571    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744574    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744577    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744579    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744582    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:09:39.751229 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744584    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744587    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744589    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744592    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744595    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744597    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744599    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744602    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744604    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744607    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744609    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744612    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744615    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744618    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744621    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744623    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744626    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744630    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744633    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:09:39.751767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744640    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:09:39.752230 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744643    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:09:39.752230 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744646    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:09:39.752230 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.744648    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:09:39.752230 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.745618    2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:09:39.752507 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.752488    2574 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 17:09:39.752544 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.752509    2574 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752558    2574 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752564    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752567    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752570    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752573    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752576    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752580    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:09:39.752580 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752582    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752585    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752589    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752594    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752597    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752600    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752603    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752606    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752608    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752611    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752613    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752616    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752619    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752622    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752624    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752627    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752629    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752632    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752636    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752638    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:09:39.752781 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752641    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752644    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752646    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752649    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752651    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752654    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752656    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752659    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752662    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752664    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752666    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752669    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752672    2574 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752675    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752678    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752681    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752683    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752686    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752689    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752692    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:09:39.753275 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752694    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752697    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752700    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752702    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752705    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752707    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752710    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752713    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752715    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752718    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752720    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752723    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752726    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752729    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752731    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752734    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752736    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752739    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752742    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:09:39.753814 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752745    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752747    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752750    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752752    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752755    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752757    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752760    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752765    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752769    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752772    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752775    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752778    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752781    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752784    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752787    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752790    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752793    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752796    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752799    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:09:39.754283 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752802    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.752807    2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752924    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752929    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752932    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752935    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752941    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752944    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752947    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752949    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752952    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752955    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752957    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752960    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752962    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752965    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 17:09:39.754767 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752968    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752970    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752973    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752975    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752979    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752981    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752984    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752987    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752989    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752993    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.752997    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753000    2574 feature_gate.go:328] unrecognized feature gate: Example2
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753003    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753006    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753009    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753012    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753015    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753018    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753021    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 17:09:39.755159 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753024    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753027    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753029    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753032    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753035    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753038    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753041    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753044    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753046    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753049    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753051    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753054    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753057    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753059    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753062    2574 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753064    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753067    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753070    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753073    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753075    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 17:09:39.755713 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753078    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753080    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753083    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753085    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753088    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753090    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753093    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753096    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753098    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753101    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753103    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753106    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753108    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753111    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753113    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753115    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753118    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753121    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753125    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 17:09:39.756199 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753129    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753132    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753135    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753138    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753140    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753143    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753145    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753148    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753150    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753153    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753156    2574 feature_gate.go:328] unrecognized feature gate: Example
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753158    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753161    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:39.753164    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.753168    2574 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 17:09:39.756736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.753284    2574 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 17:09:39.757145 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.756111    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 17:09:39.757285 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.757272    2574 server.go:1019] "Starting client certificate rotation"
May 28 17:09:39.757381 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.757366    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 17:09:39.757418 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.757395    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 17:09:39.785391 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.785371    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 17:09:39.789520 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.789500    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 17:09:39.808586 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.808562    2574 log.go:25] "Validated CRI v1 runtime API"
May 28 17:09:39.814813 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.814793    2574 log.go:25] "Validated CRI v1 image API"
May 28 17:09:39.817012 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.816994    2574 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 17:09:39.818192 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.818176    2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 17:09:39.823105 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.823083    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b6c39f3a-9763-45c3-9019-baae76a89317:/dev/nvme0n1p4 b7e21b7b-5671-455a-9a7f-27f5d1a72a95:/dev/nvme0n1p3]
May 28 17:09:39.823182 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.823104    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 17:09:39.827267 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.827163    2574 manager.go:217] Machine: {Timestamp:2026-05-28 17:09:39.826690136 +0000 UTC m=+0.460602848 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3127180 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21e2fc7a4d0c28c4b0b525b9752040 SystemUUID:ec21e2fc-7a4d-0c28-c4b0-b525b9752040 BootID:c08a1cab-ede7-44f8-b28a-569954e18ca7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:20:4f:f1:dd:2b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:20:4f:f1:dd:2b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:53:66:9f:b9:36 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 17:09:39.827892 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.827882    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 17:09:39.827979 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.827967    2574 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 17:09:39.831028 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.831006    2574 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 17:09:39.831161 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.831030    2574 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-155.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 17:09:39.831210 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.831167    2574 topology_manager.go:138] "Creating topology manager with none policy"
May 28 17:09:39.831210 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.831174    2574 container_manager_linux.go:306] "Creating device plugin manager"
May 28 17:09:39.831210 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.831182    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 17:09:39.832171 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.832158    2574 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 17:09:39.833500 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.833490    2574 state_mem.go:36] "Initialized new in-memory state store"
May 28 17:09:39.833616 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.833607    2574 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 17:09:39.836390 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.836380    2574 kubelet.go:491] "Attempting to sync node with API server"
May 28 17:09:39.836424 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.836398    2574 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 17:09:39.836424 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.836411    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 17:09:39.836424 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.836419    2574 kubelet.go:397] "Adding apiserver pod source"
May 28 17:09:39.836546 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.836428    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 17:09:39.837657 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.837642    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 17:09:39.837744 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.837661    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 17:09:39.841135 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.841119    2574 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 17:09:39.843475 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.843462    2574 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 17:09:39.845089 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845069    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845094    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845103    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845112    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845125    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845132    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845138    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845143    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845151    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845156    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 17:09:39.845172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845173    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 17:09:39.845459 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.845179    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 17:09:39.846023 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.846013    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 17:09:39.846051 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.846025    2574 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 17:09:39.849203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.849189    2574 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 17:09:39.849284 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.849222    2574 server.go:1295] "Started kubelet"
May 28 17:09:39.849374 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.849308    2574 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 17:09:39.849433 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.849392    2574 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 17:09:39.851890 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.851868    2574 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-155.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 17:09:39.852128 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.852109    2574 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 17:09:39.852243 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.852207    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 17:09:39.852303 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.852228    2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 17:09:39.852252 ip-10-0-141-155 systemd[1]: Started Kubernetes Kubelet.
May 28 17:09:39.852464 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.852298    2574 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 17:09:39.854437 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.854413    2574 server.go:317] "Adding debug handlers to kubelet server"
May 28 17:09:39.859863 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.858771    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-155.ec2.internal.18b3c9d19c4e41ee  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-155.ec2.internal,UID:ip-10-0-141-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-155.ec2.internal,},FirstTimestamp:2026-05-28 17:09:39.84920011 +0000 UTC m=+0.483112821,LastTimestamp:2026-05-28 17:09:39.84920011 +0000 UTC m=+0.483112821,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-155.ec2.internal,}"
May 28 17:09:39.862422 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862402    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 17:09:39.862502 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862421    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 17:09:39.862604 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862590    2574 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 17:09:39.862604 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862605    2574 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 17:09:39.862736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862657    2574 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 17:09:39.862736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862724    2574 reconstruct.go:97] "Volume reconstruction finished"
May 28 17:09:39.862736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.862732    2574 reconciler.go:26] "Reconciler: start to sync state"
May 28 17:09:39.863115 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863098    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 17:09:39.863115 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863114    2574 factory.go:55] Registering systemd factory
May 28 17:09:39.863115 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863120    2574 factory.go:223] Registration of the systemd container factory successfully
May 28 17:09:39.863427 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.863327    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:39.863538 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863524    2574 factory.go:153] Registering CRI-O factory
May 28 17:09:39.863605 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863543    2574 factory.go:223] Registration of the crio container factory successfully
May 28 17:09:39.863605 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863564    2574 factory.go:103] Registering Raw factory
May 28 17:09:39.863605 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.863580    2574 manager.go:1196] Started watching for new ooms in manager
May 28 17:09:39.864145 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.864115    2574 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 17:09:39.864858 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.864761    2574 manager.go:319] Starting recovery of all containers
May 28 17:09:39.872308 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.872284    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ll7lb"
May 28 17:09:39.874572 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.874417    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 17:09:39.874687 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.874664    2574 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 17:09:39.875567 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.875553    2574 manager.go:324] Recovery completed
May 28 17:09:39.879481 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.879467    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:09:39.879944 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.879930    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ll7lb"
May 28 17:09:39.881147 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.881132    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:09:39.881235 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.881156    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:09:39.881235 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.881168    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientPID"
May 28 17:09:39.881704 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.881688    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 17:09:39.881704 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.881702    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 17:09:39.881790 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.881716    2574 state_mem.go:36] "Initialized new in-memory state store"
May 28 17:09:39.883163 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.883045    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-155.ec2.internal.18b3c9d19e35b39d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-155.ec2.internal,UID:ip-10-0-141-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-155.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-155.ec2.internal,},FirstTimestamp:2026-05-28 17:09:39.881145245 +0000 UTC m=+0.515057957,LastTimestamp:2026-05-28 17:09:39.881145245 +0000 UTC m=+0.515057957,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-155.ec2.internal,}"
May 28 17:09:39.885644 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.885630    2574 policy_none.go:49] "None policy: Start"
May 28 17:09:39.885644 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.885647    2574 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 17:09:39.885748 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.885658    2574 state_mem.go:35] "Initializing new in-memory state store"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923258    2574 manager.go:341] "Starting Device Plugin manager"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.923289    2574 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923297    2574 server.go:85] "Starting device plugin registration server"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923518    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923529    2574 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923600    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923660    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.923670    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.924126    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 17:09:39.942169 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.924155    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:39.965070 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.965040    2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 17:09:39.966116 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.966102    2574 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 17:09:39.966180 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.966124    2574 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 17:09:39.966180 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.966142    2574 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 17:09:39.966180 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.966148    2574 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 17:09:39.966352 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:39.966188    2574 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 17:09:39.968662 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:39.968643    2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:09:40.024536 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.024488    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:09:40.025254 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.025242    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:09:40.025325 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.025268    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:09:40.025325 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.025279    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientPID"
May 28 17:09:40.025325 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.025305    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-155.ec2.internal"
May 28 17:09:40.035224 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.035205    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-155.ec2.internal"
May 28 17:09:40.035277 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.035230    2574 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-155.ec2.internal\": node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.046740 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.046718    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.067225 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.067206    2574 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"]
May 28 17:09:40.067299 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.067260    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:09:40.067984 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.067972    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:09:40.068045 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.067995    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:09:40.068045 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.068007    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientPID"
May 28 17:09:40.069238 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.069225    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:09:40.069876 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.069859    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:09:40.069955 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.069891    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:09:40.069955 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.069902    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientPID"
May 28 17:09:40.069955 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.069901    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.069955 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.069935    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:09:40.070567 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.070556    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:09:40.070640 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.070582    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:09:40.070640 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.070596    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientPID"
May 28 17:09:40.070990 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.070964    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.071072 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.070998    2574 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 17:09:40.071646 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.071629    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientMemory"
May 28 17:09:40.071727 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.071660    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasNoDiskPressure"
May 28 17:09:40.071727 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.071675    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeHasSufficientPID"
May 28 17:09:40.089849 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.089832    2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-155.ec2.internal\" not found" node="ip-10-0-141-155.ec2.internal"
May 28 17:09:40.093014 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.092999    2574 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-155.ec2.internal\" not found" node="ip-10-0-141-155.ec2.internal"
May 28 17:09:40.146978 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.146960    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.247807 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.247788    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.264132 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.264094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce9a07860df0679f3cff896ecb767cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal\" (UID: \"ce9a07860df0679f3cff896ecb767cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.264204 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.264137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce9a07860df0679f3cff896ecb767cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal\" (UID: \"ce9a07860df0679f3cff896ecb767cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.264204 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.264163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ebf03d5493f0f79f6e5e3abb3b6ee3b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-155.ec2.internal\" (UID: \"7ebf03d5493f0f79f6e5e3abb3b6ee3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.348537 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.348477    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.364784 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.364762    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce9a07860df0679f3cff896ecb767cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal\" (UID: \"ce9a07860df0679f3cff896ecb767cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.364848 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.364789    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce9a07860df0679f3cff896ecb767cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal\" (UID: \"ce9a07860df0679f3cff896ecb767cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.364848 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.364807    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ebf03d5493f0f79f6e5e3abb3b6ee3b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-155.ec2.internal\" (UID: \"7ebf03d5493f0f79f6e5e3abb3b6ee3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.364848 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.364836    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7ebf03d5493f0f79f6e5e3abb3b6ee3b-config\") pod \"kube-apiserver-proxy-ip-10-0-141-155.ec2.internal\" (UID: \"7ebf03d5493f0f79f6e5e3abb3b6ee3b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.364965 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.364868    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce9a07860df0679f3cff896ecb767cf5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal\" (UID: \"ce9a07860df0679f3cff896ecb767cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.364965 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.364870    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce9a07860df0679f3cff896ecb767cf5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal\" (UID: \"ce9a07860df0679f3cff896ecb767cf5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.392896 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.392864    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.395479 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.395465    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"
May 28 17:09:40.449239 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.449215    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.549724 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.549695    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.650217 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.650152    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.750680 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.750649    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.756809 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.756789    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 17:09:40.756950 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.756935    2574 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 17:09:40.796263 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.796243    2574 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:09:40.843375 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.843357    2574 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:09:40.851583 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.851555    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.863320 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.863301    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 17:09:40.874722 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.874696    2574 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 17:09:40.882780 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.882751    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 17:04:39 +0000 UTC" deadline="2027-12-24 22:54:29.217689681 +0000 UTC"
May 28 17:09:40.882780 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.882778    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13805h44m48.33491463s"
May 28 17:09:40.925540 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.925491    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zgm9k"
May 28 17:09:40.932222 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.932204    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zgm9k"
May 28 17:09:40.951698 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:40.951674    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:40.976018 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:40.975976    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9a07860df0679f3cff896ecb767cf5.slice/crio-401ed5c20cf9c00922c442dc878dd8060144764f271c5cdeaa82a2f73ed1a182 WatchSource:0}: Error finding container 401ed5c20cf9c00922c442dc878dd8060144764f271c5cdeaa82a2f73ed1a182: Status 404 returned error can't find the container with id 401ed5c20cf9c00922c442dc878dd8060144764f271c5cdeaa82a2f73ed1a182
May 28 17:09:40.976232 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:40.976217    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebf03d5493f0f79f6e5e3abb3b6ee3b.slice/crio-c290c9cd8f050439bd700df031d91828cbe83b8b5d5848a64d732715f583d961 WatchSource:0}: Error finding container c290c9cd8f050439bd700df031d91828cbe83b8b5d5848a64d732715f583d961: Status 404 returned error can't find the container with id c290c9cd8f050439bd700df031d91828cbe83b8b5d5848a64d732715f583d961
May 28 17:09:40.980938 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:40.980925    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:09:41.052409 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.052388    2574 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-141-155.ec2.internal\" not found"
May 28 17:09:41.127304 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.127282    2574 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:09:41.162838 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.162816    2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal"
May 28 17:09:41.177309 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.177238    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 17:09:41.178391 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.178378    2574 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal"
May 28 17:09:41.186708 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.186694    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 17:09:41.614813 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.614777    2574 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 17:09:41.837981 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.837894    2574 apiserver.go:52] "Watching apiserver"
May 28 17:09:41.844124 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.844103    2574 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 17:09:41.846863 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.846832    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-qpgcw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal","openshift-multus/network-metrics-daemon-j9xtb","openshift-network-diagnostics/network-check-target-sdjr4","openshift-ovn-kubernetes/ovnkube-node-nxzl5","openshift-dns/node-resolver-bvsbc","openshift-image-registry/node-ca-7mc88","openshift-multus/multus-additional-cni-plugins-s9w8f","openshift-multus/multus-d9w6x","openshift-network-operator/iptables-alerter-hv7cn","kube-system/konnectivity-agent-dtzs2","kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"]
May 28 17:09:41.850113 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.850093    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.852179 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.852155    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:41.852304 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.852232    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:41.852791 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.852769    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 17:09:41.852891 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.852867    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 17:09:41.852970 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.852946    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 17:09:41.852970 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.852959    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kbrwr\""
May 28 17:09:41.854264 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.854248    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:41.854365 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.854330    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:41.856761 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.856739    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.858706 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.858688    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.860188 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.860168    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 17:09:41.860366 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.860324    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l9mxl\""
May 28 17:09:41.860710 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.860687    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 17:09:41.860793 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.860738    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 17:09:41.860947 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.860930    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 17:09:41.861037 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.860947    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.861706 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.861509    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 17:09:41.861706 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.861585    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 17:09:41.861706 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.861633    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 17:09:41.861907 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.861786    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 17:09:41.863153 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.863134    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5rrpq\""
May 28 17:09:41.864513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.863827    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cnt49\""
May 28 17:09:41.864513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.864247    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 17:09:41.864513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.864270    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 17:09:41.868044 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.866112    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.869421 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.869378    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 17:09:41.869522 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.869434    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 17:09:41.869924 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.869848    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 17:09:41.870010 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.869957    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 17:09:41.870010 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.869987    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 17:09:41.870305 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870129    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wv65s\""
May 28 17:09:41.870305 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870254    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.870668 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovnkube-config\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.870668 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870664    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovnkube-script-lib\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.870812 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-cnibin\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.870812 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870776    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-kubernetes\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.870895 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870819    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysctl-conf\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.870895 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870875    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-lib-modules\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.870995 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7a823a-75e6-4766-aba1-2ca7274f1512-tmp\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.870995 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870945    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-kubelet\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.870995 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870970    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-systemd\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.870995 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.870985    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-tmp-dir\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871032    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-cni-binary-copy\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871048    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-modprobe-d\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-run-netns\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-var-lib-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.871184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871166    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-ovn\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.871614 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871238    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.871614 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.871614 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysconfig\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.871614 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871313    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-systemd\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.871830 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871335    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-host\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.871830 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:41.871914 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871843    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-etc-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.871914 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-system-cni-dir\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.871914 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871892    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-run\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872010 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871916    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-sys\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872010 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871940    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:41.872010 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871963    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-systemd-units\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872010 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.871987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-slash\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872182 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872012    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-node-log\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872182 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872034    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-log-socket\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872182 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmqlm\" (UniqueName: \"kubernetes.io/projected/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-kube-api-access-cmqlm\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.872182 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872086    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n72z\" (UniqueName: \"kubernetes.io/projected/4ba637e9-8b45-407d-8e68-50133573feac-kube-api-access-2n72z\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.872182 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872150    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-host\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872185    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkkjf\" (UniqueName: \"kubernetes.io/projected/0f7a823a-75e6-4766-aba1-2ca7274f1512-kube-api-access-pkkjf\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-serviceca\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872226    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzthw\" (UniqueName: \"kubernetes.io/projected/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-kube-api-access-xzthw\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872270    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovn-node-metrics-cert\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n78zp\" (UniqueName: \"kubernetes.io/projected/98543425-257a-4fa1-b5ad-fbef8f1e40b6-kube-api-access-n78zp\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872371    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-hosts-file\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872395    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-os-release\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysctl-d\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872442 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-cni-bin\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872472    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872605    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872611    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872645    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-env-overrides\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872669    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-var-lib-kubelet\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872681    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7bs92\""
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872690    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-tuned\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872714    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncdkf\" (UniqueName: \"kubernetes.io/projected/3a5cb798-cc5b-4fdf-9671-67201a9e7718-kube-api-access-ncdkf\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:41.872957 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.872740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-cni-netd\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.874988 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.874968    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 17:09:41.875268 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.874999    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 17:09:41.875268 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.875016    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9vt8g\""
May 28 17:09:41.875268 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.875152    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:41.875268 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.875240    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.876008 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.875872    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 17:09:41.877977 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.877574    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-skhb2\""
May 28 17:09:41.877977 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.877613    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 17:09:41.877977 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.877807    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 17:09:41.877977 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.877831    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 17:09:41.877977 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.877871    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 17:09:41.878225 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.878094    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-brvct\""
May 28 17:09:41.878225 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.878092    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 17:09:41.933837 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.933807    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:04:40 +0000 UTC" deadline="2027-11-13 05:02:09.96252564 +0000 UTC"
May 28 17:09:41.933837 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.933831    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12803h52m28.02869676s"
May 28 17:09:41.964539 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.964515    2574 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 17:09:41.972089 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.972043    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal" event={"ID":"ce9a07860df0679f3cff896ecb767cf5","Type":"ContainerStarted","Data":"401ed5c20cf9c00922c442dc878dd8060144764f271c5cdeaa82a2f73ed1a182"}
May 28 17:09:41.972906 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.972883    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-iptables-alerter-script\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:41.973000 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.972926    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-os-release\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.973000 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.972953    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysctl-d\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973000 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.972980    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-var-lib-kubelet\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973005    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-tuned\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973030    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ncdkf\" (UniqueName: \"kubernetes.io/projected/3a5cb798-cc5b-4fdf-9671-67201a9e7718-kube-api-access-ncdkf\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973036    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-os-release\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-daemon-config\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcbp7\" (UniqueName: \"kubernetes.io/projected/860d8d43-a916-4d50-ac0c-7e0e927b00e3-kube-api-access-rcbp7\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973100    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-cnibin\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973120    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysctl-conf\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973145    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-kubelet\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973155    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysctl-d\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973170    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-systemd\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973194    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973216    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-ovn\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973240    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-cnibin\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973240    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal" event={"ID":"7ebf03d5493f0f79f6e5e3abb3b6ee3b","Type":"ContainerStarted","Data":"c290c9cd8f050439bd700df031d91828cbe83b8b5d5848a64d732715f583d961"}
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973270    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-var-lib-kubelet\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973264    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-k8s-cni-cncf-io\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973331    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-systemd\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973335    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973390    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-cni-netd\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973392    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973389    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973396    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysctl-conf\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973420    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-run-netns\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973442    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-cnibin\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973445    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-var-lib-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973458    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-run-ovn\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.973650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973356    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-kubelet\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-cni-multus\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973511    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-multus-certs\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973513    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973527    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-run-netns\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jsqj\" (UniqueName: \"kubernetes.io/projected/4e72d728-8f82-4fd9-b8cf-0377519b2328-kube-api-access-7jsqj\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973484    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-var-lib-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973563    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovn-node-metrics-cert\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973590    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973601    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-cni-netd\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973618    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973646    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-systemd\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973667    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-log-socket\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973693    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-etc-kubernetes\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-869gx\" (UniqueName: \"kubernetes.io/projected/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-kube-api-access-869gx\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973746    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-cni-bin\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973754    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.976214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973773    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-systemd\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973816    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-env-overrides\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973842    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-system-cni-dir\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973869    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-socket-dir-parent\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973874    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973895    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d08de4f8-cf60-48a9-b73a-6d41594a9b61-konnectivity-ca\") pod \"konnectivity-agent-dtzs2\" (UID: \"d08de4f8-cf60-48a9-b73a-6d41594a9b61\") " pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973926    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cmqlm\" (UniqueName: \"kubernetes.io/projected/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-kube-api-access-cmqlm\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973960    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2n72z\" (UniqueName: \"kubernetes.io/projected/4ba637e9-8b45-407d-8e68-50133573feac-kube-api-access-2n72z\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973966    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4ba637e9-8b45-407d-8e68-50133573feac-system-cni-dir\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973994    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-host\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974021    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-serviceca\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974045    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xzthw\" (UniqueName: \"kubernetes.io/projected/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-kube-api-access-xzthw\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974070    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n78zp\" (UniqueName: \"kubernetes.io/projected/98543425-257a-4fa1-b5ad-fbef8f1e40b6-kube-api-access-n78zp\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-socket-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974132    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-hosts-file\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974168    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-kubelet\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.977041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974201    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovnkube-config\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-conf-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974261    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-host-slash\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974273    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-device-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.973929    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-cni-bin\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974325    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974368    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-sys-fs\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974419    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-log-socket\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974449    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-system-cni-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974512    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-kubernetes\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974538    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-lib-modules\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974563    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7a823a-75e6-4766-aba1-2ca7274f1512-tmp\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974588    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-node-log\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-os-release\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974687    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovnkube-script-lib\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974720    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-tmp-dir\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.977865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974750    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-cni-binary-copy\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974776    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974803    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-modprobe-d\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974822    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-serviceca\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974842    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974865    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-registration-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974863    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-env-overrides\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974890    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysconfig\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974923    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-lib-modules\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974966    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-host\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974971    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-kubernetes\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.974997    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975012    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-host\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975023    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-etc-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975049    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-run\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975053    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-sysconfig\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975073    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-sys\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975120    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-hosts-file\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.979063 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975171    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-host\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975367    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975423    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-systemd-units\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975451    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-slash\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975480    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e72d728-8f82-4fd9-b8cf-0377519b2328-cni-binary-copy\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-cni-bin\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975522    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975535    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pkkjf\" (UniqueName: \"kubernetes.io/projected/0f7a823a-75e6-4766-aba1-2ca7274f1512-kube-api-access-pkkjf\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975579    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-sys\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-netns\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975611    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4ba637e9-8b45-407d-8e68-50133573feac-cni-binary-copy\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975628    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-hostroot\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975675    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-systemd-units\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975778    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-host-slash\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975846    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-run\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.975904    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975914    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-tmp-dir\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.979556 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.975994    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-etc-openvswitch\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.976020    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:42.475952907 +0000 UTC m=+3.109865627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.976066    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98543425-257a-4fa1-b5ad-fbef8f1e40b6-node-log\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.976103    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d08de4f8-cf60-48a9-b73a-6d41594a9b61-agent-certs\") pod \"konnectivity-agent-dtzs2\" (UID: \"d08de4f8-cf60-48a9-b73a-6d41594a9b61\") " pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.976130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-cni-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.976242    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-modprobe-d\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.976796    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovnkube-config\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.976974    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0f7a823a-75e6-4766-aba1-2ca7274f1512-etc-tuned\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.977081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovn-node-metrics-cert\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.978723    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98543425-257a-4fa1-b5ad-fbef8f1e40b6-ovnkube-script-lib\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:41.980203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.978882    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f7a823a-75e6-4766-aba1-2ca7274f1512-tmp\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.984561 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.984541    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:09:41.984659 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.984566    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:09:41.984659 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.984581    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:41.984659 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:41.984651    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:42.484635383 +0000 UTC m=+3.118548101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:41.987514 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.987492    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkkjf\" (UniqueName: \"kubernetes.io/projected/0f7a823a-75e6-4766-aba1-2ca7274f1512-kube-api-access-pkkjf\") pod \"tuned-qpgcw\" (UID: \"0f7a823a-75e6-4766-aba1-2ca7274f1512\") " pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:41.987611 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.987492    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n72z\" (UniqueName: \"kubernetes.io/projected/4ba637e9-8b45-407d-8e68-50133573feac-kube-api-access-2n72z\") pod \"multus-additional-cni-plugins-s9w8f\" (UID: \"4ba637e9-8b45-407d-8e68-50133573feac\") " pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:41.988187 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.987913    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmqlm\" (UniqueName: \"kubernetes.io/projected/fd0fc226-3bdc-4e62-b9d4-0ccadcc58379-kube-api-access-cmqlm\") pod \"node-resolver-bvsbc\" (UID: \"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379\") " pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:41.988187 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.987930    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncdkf\" (UniqueName: \"kubernetes.io/projected/3a5cb798-cc5b-4fdf-9671-67201a9e7718-kube-api-access-ncdkf\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:41.988314 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.988190    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzthw\" (UniqueName: \"kubernetes.io/projected/a8cc9d5d-aa7b-42f4-aa40-11579b736fa8-kube-api-access-xzthw\") pod \"node-ca-7mc88\" (UID: \"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8\") " pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:41.988449 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:41.988404    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n78zp\" (UniqueName: \"kubernetes.io/projected/98543425-257a-4fa1-b5ad-fbef8f1e40b6-kube-api-access-n78zp\") pod \"ovnkube-node-nxzl5\" (UID: \"98543425-257a-4fa1-b5ad-fbef8f1e40b6\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:42.077221 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077190    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-socket-dir-parent\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077221 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077224    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d08de4f8-cf60-48a9-b73a-6d41594a9b61-konnectivity-ca\") pod \"konnectivity-agent-dtzs2\" (UID: \"d08de4f8-cf60-48a9-b73a-6d41594a9b61\") " pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077243    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-socket-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077259    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-kubelet\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077274    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-conf-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077296    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-host-slash\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077321    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-socket-dir-parent\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077355    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-kubelet\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077358    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-host-slash\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077379    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-device-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077401    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-conf-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077409    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077422    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-socket-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077426    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-device-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077445    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-sys-fs\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077489    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-sys-fs\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077494    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-etc-selinux\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077496    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-system-cni-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077534    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-os-release\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077545    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-system-cni-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077565    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077590    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-registration-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077605    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-os-release\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077606    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077642    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e72d728-8f82-4fd9-b8cf-0377519b2328-cni-binary-copy\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077659    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/860d8d43-a916-4d50-ac0c-7e0e927b00e3-registration-dir\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077667    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-cni-bin\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.077815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077700    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-netns\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077724    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-hostroot\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077750    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d08de4f8-cf60-48a9-b73a-6d41594a9b61-agent-certs\") pod \"konnectivity-agent-dtzs2\" (UID: \"d08de4f8-cf60-48a9-b73a-6d41594a9b61\") " pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077753    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d08de4f8-cf60-48a9-b73a-6d41594a9b61-konnectivity-ca\") pod \"konnectivity-agent-dtzs2\" (UID: \"d08de4f8-cf60-48a9-b73a-6d41594a9b61\") " pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077766    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-netns\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077700    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-cni-bin\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077798    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-hostroot\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077775    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-cni-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077836    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-iptables-alerter-script\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077861    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-daemon-config\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077887    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rcbp7\" (UniqueName: \"kubernetes.io/projected/860d8d43-a916-4d50-ac0c-7e0e927b00e3-kube-api-access-rcbp7\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077896    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-cni-dir\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077916    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-cnibin\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077940    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-k8s-cni-cncf-io\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.077988    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-cni-multus\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078011    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-multus-certs\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078040    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7jsqj\" (UniqueName: \"kubernetes.io/projected/4e72d728-8f82-4fd9-b8cf-0377519b2328-kube-api-access-7jsqj\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078068    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-etc-kubernetes\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.078582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078100    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-869gx\" (UniqueName: \"kubernetes.io/projected/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-kube-api-access-869gx\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078132    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-var-lib-cni-multus\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078177    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e72d728-8f82-4fd9-b8cf-0377519b2328-cni-binary-copy\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078314    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-cnibin\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-multus-certs\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078377    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-host-run-k8s-cni-cncf-io\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078405    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e72d728-8f82-4fd9-b8cf-0377519b2328-etc-kubernetes\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.078410    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4e72d728-8f82-4fd9-b8cf-0377519b2328-multus-daemon-config\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.079276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.079108    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-iptables-alerter-script\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.080383 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.080365    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d08de4f8-cf60-48a9-b73a-6d41594a9b61-agent-certs\") pod \"konnectivity-agent-dtzs2\" (UID: \"d08de4f8-cf60-48a9-b73a-6d41594a9b61\") " pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:42.088865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.088811    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcbp7\" (UniqueName: \"kubernetes.io/projected/860d8d43-a916-4d50-ac0c-7e0e927b00e3-kube-api-access-rcbp7\") pod \"aws-ebs-csi-driver-node-5kx5h\" (UID: \"860d8d43-a916-4d50-ac0c-7e0e927b00e3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.089184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.089152    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-869gx\" (UniqueName: \"kubernetes.io/projected/3089b2b0-3c18-4a11-b853-3cdb8fcf459a-kube-api-access-869gx\") pod \"iptables-alerter-hv7cn\" (UID: \"3089b2b0-3c18-4a11-b853-3cdb8fcf459a\") " pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.089278 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.089225    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jsqj\" (UniqueName: \"kubernetes.io/projected/4e72d728-8f82-4fd9-b8cf-0377519b2328-kube-api-access-7jsqj\") pod \"multus-d9w6x\" (UID: \"4e72d728-8f82-4fd9-b8cf-0377519b2328\") " pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.159065 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.159034    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7mc88"
May 28 17:09:42.169826 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.169803    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:09:42.182596 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.182572    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bvsbc"
May 28 17:09:42.187205 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.187177    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qpgcw"
May 28 17:09:42.194842 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.194812    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s9w8f"
May 28 17:09:42.201446 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.201427    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d9w6x"
May 28 17:09:42.208892 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.208874    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hv7cn"
May 28 17:09:42.217391 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.217372    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:42.222851 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.222834    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h"
May 28 17:09:42.481207 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.481115    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:42.481383 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:42.481277    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:42.481383 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:42.481361    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:43.481328454 +0000 UTC m=+4.115241156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:42.581780 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.581748    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:42.581958 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:42.581870    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:09:42.581958 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:42.581886    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:09:42.581958 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:42.581894    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:42.581958 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:42.581946    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:43.581925461 +0000 UTC m=+4.215838162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:42.699917 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.699886    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8cc9d5d_aa7b_42f4_aa40_11579b736fa8.slice/crio-5cde57d9698d0fe728cc7a0ca44d694a758c6468a5e54e79d57955d793198dbe WatchSource:0}: Error finding container 5cde57d9698d0fe728cc7a0ca44d694a758c6468a5e54e79d57955d793198dbe: Status 404 returned error can't find the container with id 5cde57d9698d0fe728cc7a0ca44d694a758c6468a5e54e79d57955d793198dbe
May 28 17:09:42.703214 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.703184    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e72d728_8f82_4fd9_b8cf_0377519b2328.slice/crio-0d8f7312a06073102a0dd5581dd80dbe74aed38cb194bdf4a2d433ff3e2f5728 WatchSource:0}: Error finding container 0d8f7312a06073102a0dd5581dd80dbe74aed38cb194bdf4a2d433ff3e2f5728: Status 404 returned error can't find the container with id 0d8f7312a06073102a0dd5581dd80dbe74aed38cb194bdf4a2d433ff3e2f5728
May 28 17:09:42.706136 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.706076    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08de4f8_cf60_48a9_b73a_6d41594a9b61.slice/crio-ce81b46ed619225fc9b887620564220895954440f8c1128727c1e2d1d90a03b7 WatchSource:0}: Error finding container ce81b46ed619225fc9b887620564220895954440f8c1128727c1e2d1d90a03b7: Status 404 returned error can't find the container with id ce81b46ed619225fc9b887620564220895954440f8c1128727c1e2d1d90a03b7
May 28 17:09:42.707711 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.707692    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98543425_257a_4fa1_b5ad_fbef8f1e40b6.slice/crio-c153fbec63735892fd37381d458caa52d6213b3dea10b8c27dd56ce806d4b0b0 WatchSource:0}: Error finding container c153fbec63735892fd37381d458caa52d6213b3dea10b8c27dd56ce806d4b0b0: Status 404 returned error can't find the container with id c153fbec63735892fd37381d458caa52d6213b3dea10b8c27dd56ce806d4b0b0
May 28 17:09:42.708462 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.708439    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f7a823a_75e6_4766_aba1_2ca7274f1512.slice/crio-9e2b905764391081af836443c7b9e599c34ba6ec1bef4ea0090ad46796514f8c WatchSource:0}: Error finding container 9e2b905764391081af836443c7b9e599c34ba6ec1bef4ea0090ad46796514f8c: Status 404 returned error can't find the container with id 9e2b905764391081af836443c7b9e599c34ba6ec1bef4ea0090ad46796514f8c
May 28 17:09:42.710136 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.710089    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba637e9_8b45_407d_8e68_50133573feac.slice/crio-be4ab92d172b317174a882bdeef472579e1e4e08b836fe3e09d702fad29e9e0e WatchSource:0}: Error finding container be4ab92d172b317174a882bdeef472579e1e4e08b836fe3e09d702fad29e9e0e: Status 404 returned error can't find the container with id be4ab92d172b317174a882bdeef472579e1e4e08b836fe3e09d702fad29e9e0e
May 28 17:09:42.711892 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.711874    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0fc226_3bdc_4e62_b9d4_0ccadcc58379.slice/crio-e760191ec3b1dc34ad565f35c1ec70bda8fa6aa994982428895fff6f4c74c3aa WatchSource:0}: Error finding container e760191ec3b1dc34ad565f35c1ec70bda8fa6aa994982428895fff6f4c74c3aa: Status 404 returned error can't find the container with id e760191ec3b1dc34ad565f35c1ec70bda8fa6aa994982428895fff6f4c74c3aa
May 28 17:09:42.732108 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.732079    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3089b2b0_3c18_4a11_b853_3cdb8fcf459a.slice/crio-f04293def65f522e779d8ce761f4ee7628d8d222ac96145f9de5e3e9cc67434c WatchSource:0}: Error finding container f04293def65f522e779d8ce761f4ee7628d8d222ac96145f9de5e3e9cc67434c: Status 404 returned error can't find the container with id f04293def65f522e779d8ce761f4ee7628d8d222ac96145f9de5e3e9cc67434c
May 28 17:09:42.732921 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:09:42.732900    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860d8d43_a916_4d50_ac0c_7e0e927b00e3.slice/crio-3e15863be78b17490e03f854e110174f9180636870aadab9f0fbaed8bf9273e5 WatchSource:0}: Error finding container 3e15863be78b17490e03f854e110174f9180636870aadab9f0fbaed8bf9273e5: Status 404 returned error can't find the container with id 3e15863be78b17490e03f854e110174f9180636870aadab9f0fbaed8bf9273e5
May 28 17:09:42.934758 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.934557    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 17:04:40 +0000 UTC" deadline="2028-01-18 23:15:38.02132909 +0000 UTC"
May 28 17:09:42.934758 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.934756    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14406h5m55.086578028s"
May 28 17:09:42.976741 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.976712    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal" event={"ID":"7ebf03d5493f0f79f6e5e3abb3b6ee3b","Type":"ContainerStarted","Data":"4450392749fc116db3fbb19ec67b3b99b8572133c43c83b9b61d5e412ed2258a"}
May 28 17:09:42.977842 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.977815    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerStarted","Data":"be4ab92d172b317174a882bdeef472579e1e4e08b836fe3e09d702fad29e9e0e"}
May 28 17:09:42.978821 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.978792    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"c153fbec63735892fd37381d458caa52d6213b3dea10b8c27dd56ce806d4b0b0"}
May 28 17:09:42.979776 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.979757    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d9w6x" event={"ID":"4e72d728-8f82-4fd9-b8cf-0377519b2328","Type":"ContainerStarted","Data":"0d8f7312a06073102a0dd5581dd80dbe74aed38cb194bdf4a2d433ff3e2f5728"}
May 28 17:09:42.981018 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.980995    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7mc88" event={"ID":"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8","Type":"ContainerStarted","Data":"5cde57d9698d0fe728cc7a0ca44d694a758c6468a5e54e79d57955d793198dbe"}
May 28 17:09:42.982320 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.982207    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h" event={"ID":"860d8d43-a916-4d50-ac0c-7e0e927b00e3","Type":"ContainerStarted","Data":"3e15863be78b17490e03f854e110174f9180636870aadab9f0fbaed8bf9273e5"}
May 28 17:09:42.983714 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.983690    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hv7cn" event={"ID":"3089b2b0-3c18-4a11-b853-3cdb8fcf459a","Type":"ContainerStarted","Data":"f04293def65f522e779d8ce761f4ee7628d8d222ac96145f9de5e3e9cc67434c"}
May 28 17:09:42.984791 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.984770    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bvsbc" event={"ID":"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379","Type":"ContainerStarted","Data":"e760191ec3b1dc34ad565f35c1ec70bda8fa6aa994982428895fff6f4c74c3aa"}
May 28 17:09:42.985726 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.985701    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qpgcw" event={"ID":"0f7a823a-75e6-4766-aba1-2ca7274f1512","Type":"ContainerStarted","Data":"9e2b905764391081af836443c7b9e599c34ba6ec1bef4ea0090ad46796514f8c"}
May 28 17:09:42.986506 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:42.986485    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dtzs2" event={"ID":"d08de4f8-cf60-48a9-b73a-6d41594a9b61","Type":"ContainerStarted","Data":"ce81b46ed619225fc9b887620564220895954440f8c1128727c1e2d1d90a03b7"}
May 28 17:09:43.487127 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:43.487093    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:43.487302 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.487281    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:43.487387 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.487370    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:45.487335027 +0000 UTC m=+6.121247739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:43.589016 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:43.588376    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:43.589016 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.588545    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:09:43.589016 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.588563    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:09:43.589016 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.588576    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:43.589016 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.588633    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:45.588615723 +0000 UTC m=+6.222528436 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:43.969590 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:43.967182    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:43.969590 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.967316    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:43.969590 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:43.967836    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:43.969590 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:43.967933    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:44.010787 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:44.010753    2574 generic.go:358] "Generic (PLEG): container finished" podID="ce9a07860df0679f3cff896ecb767cf5" containerID="3504657a38cf2bebad89cfec6116bc05d03aa679249536deb3d61624e498aa43" exitCode=0
May 28 17:09:44.011259 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:44.011236    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal" event={"ID":"ce9a07860df0679f3cff896ecb767cf5","Type":"ContainerDied","Data":"3504657a38cf2bebad89cfec6116bc05d03aa679249536deb3d61624e498aa43"}
May 28 17:09:44.026753 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:44.026708    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-155.ec2.internal" podStartSLOduration=3.02669244 podStartE2EDuration="3.02669244s" podCreationTimestamp="2026-05-28 17:09:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:09:42.990105856 +0000 UTC m=+3.624018560" watchObservedRunningTime="2026-05-28 17:09:44.02669244 +0000 UTC m=+4.660605164"
May 28 17:09:45.025364 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:45.025236    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal" event={"ID":"ce9a07860df0679f3cff896ecb767cf5","Type":"ContainerStarted","Data":"e3c7af512e6cc2d42f0520861c9411aa6512a750a467aec7439c934171c1fc9c"}
May 28 17:09:45.505809 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:45.505191    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:45.505809 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.505370    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:45.505809 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.505437    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:49.505419924 +0000 UTC m=+10.139332637 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:45.605631 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:45.605577    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:45.605807 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.605773    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:09:45.605807 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.605794    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:09:45.605807 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.605807    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:45.605963 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.605865    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:49.605847208 +0000 UTC m=+10.239759910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:45.966840 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:45.966757    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:45.967003 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.966900    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:45.967279 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:45.967256    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:45.967488 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:45.967398    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:47.967357 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:47.967309    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:47.967786 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:47.967374    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:47.967786 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:47.967455    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:47.967786 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:47.967546    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:49.533405 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:49.533362    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:49.533837 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.533466    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:49.533837 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.533545    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:57.533522752 +0000 UTC m=+18.167435452 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:49.634490 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:49.634449    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:49.634670 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.634631    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:09:49.634670 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.634655    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:09:49.634670 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.634670    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:49.634821 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.634736    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:57.634716547 +0000 UTC m=+18.268629261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:49.968889 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:49.968325    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:49.968889 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.968469    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:49.969318 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:49.969188    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:49.969318 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:49.969283    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:50.838286 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:50.838233    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-155.ec2.internal" podStartSLOduration=9.838215702 podStartE2EDuration="9.838215702s" podCreationTimestamp="2026-05-28 17:09:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:09:45.042252689 +0000 UTC m=+5.676165410" watchObservedRunningTime="2026-05-28 17:09:50.838215702 +0000 UTC m=+11.472128423"
May 28 17:09:50.839007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:50.838987    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tc52b"]
May 28 17:09:50.844703 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:50.844683    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:50.844798 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:50.844747    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:09:50.944285 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:50.944250    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741cdce6-f910-45f0-8929-0d0bf2be6442-dbus\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:50.944475 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:50.944298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:50.944475 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:50.944367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741cdce6-f910-45f0-8929-0d0bf2be6442-kubelet-config\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.045716 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.045664    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741cdce6-f910-45f0-8929-0d0bf2be6442-kubelet-config\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.045902 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.045753    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741cdce6-f910-45f0-8929-0d0bf2be6442-dbus\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.045902 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.045781    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.045902 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.045780    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/741cdce6-f910-45f0-8929-0d0bf2be6442-kubelet-config\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.045902 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.045890    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:09:51.046108 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.045930    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/741cdce6-f910-45f0-8929-0d0bf2be6442-dbus\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.046108 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.045947    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret podName:741cdce6-f910-45f0-8929-0d0bf2be6442 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:51.545927661 +0000 UTC m=+12.179840363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret") pod "global-pull-secret-syncer-tc52b" (UID: "741cdce6-f910-45f0-8929-0d0bf2be6442") : object "kube-system"/"original-pull-secret" not registered
May 28 17:09:51.549727 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.549668    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.549885 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.549833    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:09:51.549939 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.549896    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret podName:741cdce6-f910-45f0-8929-0d0bf2be6442 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:52.549875965 +0000 UTC m=+13.183788678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret") pod "global-pull-secret-syncer-tc52b" (UID: "741cdce6-f910-45f0-8929-0d0bf2be6442") : object "kube-system"/"original-pull-secret" not registered
May 28 17:09:51.966829 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.966736    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:51.966829 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.966807    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:51.967318 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:51.966945    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:51.967318 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.966938    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:09:51.967318 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.967058    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:51.967318 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:51.967143    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:52.557243 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:52.557210    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:52.557429 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:52.557365    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:09:52.557487 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:52.557433    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret podName:741cdce6-f910-45f0-8929-0d0bf2be6442 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:54.557413417 +0000 UTC m=+15.191326139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret") pod "global-pull-secret-syncer-tc52b" (UID: "741cdce6-f910-45f0-8929-0d0bf2be6442") : object "kube-system"/"original-pull-secret" not registered
May 28 17:09:53.041653 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.041619    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7mc88" event={"ID":"a8cc9d5d-aa7b-42f4-aa40-11579b736fa8","Type":"ContainerStarted","Data":"e0d55a4a519657fbf5d2cf26e65e818bb04b48a7c7a2cee101634aeaabe73b3d"}
May 28 17:09:53.044453 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.044426    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h" event={"ID":"860d8d43-a916-4d50-ac0c-7e0e927b00e3","Type":"ContainerStarted","Data":"909b9fbbdac02633c773d1b345d6e96705ac231721da51b3cfe54eb97c909c39"}
May 28 17:09:53.046320 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.046298    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qpgcw" event={"ID":"0f7a823a-75e6-4766-aba1-2ca7274f1512","Type":"ContainerStarted","Data":"a77bc3f5b99a97d03403d9affe59d36573c082575bbf5840d8204bb17acc35ad"}
May 28 17:09:53.048765 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.048746    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dtzs2" event={"ID":"d08de4f8-cf60-48a9-b73a-6d41594a9b61","Type":"ContainerStarted","Data":"79260fe4d98f52bbfba618f6506a93010389b372c34842b6fa070b7116c549f0"}
May 28 17:09:53.060067 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.059710    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7mc88" podStartSLOduration=3.026997656 podStartE2EDuration="13.059693851s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.703749304 +0000 UTC m=+3.337662003" lastFinishedPulling="2026-05-28 17:09:52.736445483 +0000 UTC m=+13.370358198" observedRunningTime="2026-05-28 17:09:53.059493421 +0000 UTC m=+13.693406142" watchObservedRunningTime="2026-05-28 17:09:53.059693851 +0000 UTC m=+13.693606572"
May 28 17:09:53.076943 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.076766    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dtzs2" podStartSLOduration=3.020019729 podStartE2EDuration="13.076752115s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.707890376 +0000 UTC m=+3.341803074" lastFinishedPulling="2026-05-28 17:09:52.764622747 +0000 UTC m=+13.398535460" observedRunningTime="2026-05-28 17:09:53.076740017 +0000 UTC m=+13.710652739" watchObservedRunningTime="2026-05-28 17:09:53.076752115 +0000 UTC m=+13.710664835"
May 28 17:09:53.092831 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.092105    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qpgcw" podStartSLOduration=3.037338761 podStartE2EDuration="13.092090713s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.710414729 +0000 UTC m=+3.344327446" lastFinishedPulling="2026-05-28 17:09:52.765166693 +0000 UTC m=+13.399079398" observedRunningTime="2026-05-28 17:09:53.091951661 +0000 UTC m=+13.725864406" watchObservedRunningTime="2026-05-28 17:09:53.092090713 +0000 UTC m=+13.726003434"
May 28 17:09:53.967152 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.967122    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:53.967374 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.967122    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:53.967374 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:53.967261    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:53.967374 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:53.967122    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:53.967557 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:53.967317    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:09:53.967557 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:53.967433    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:54.051407 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:54.051370    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bvsbc" event={"ID":"fd0fc226-3bdc-4e62-b9d4-0ccadcc58379","Type":"ContainerStarted","Data":"211ac5ce43424aaa0feb3289313f1dc3ec0519a79216d5c18336e3d0548a9374"}
May 28 17:09:54.052959 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:54.052935    2574 generic.go:358] "Generic (PLEG): container finished" podID="4ba637e9-8b45-407d-8e68-50133573feac" containerID="e4504da375afd482fdf21749819d0f5e73069664a4745b1e504a1be1b9da833a" exitCode=0
May 28 17:09:54.053080 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:54.053043    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerDied","Data":"e4504da375afd482fdf21749819d0f5e73069664a4745b1e504a1be1b9da833a"}
May 28 17:09:54.066187 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:54.066133    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-bvsbc" podStartSLOduration=4.028010444 podStartE2EDuration="14.066117527s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.731152584 +0000 UTC m=+3.365065296" lastFinishedPulling="2026-05-28 17:09:52.769259678 +0000 UTC m=+13.403172379" observedRunningTime="2026-05-28 17:09:54.065429115 +0000 UTC m=+14.699341840" watchObservedRunningTime="2026-05-28 17:09:54.066117527 +0000 UTC m=+14.700030250"
May 28 17:09:54.573088 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:54.573013    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:54.573221 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:54.573148    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:09:54.573221 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:54.573216    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret podName:741cdce6-f910-45f0-8929-0d0bf2be6442 nodeName:}" failed. No retries permitted until 2026-05-28 17:09:58.573194986 +0000 UTC m=+19.207107698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret") pod "global-pull-secret-syncer-tc52b" (UID: "741cdce6-f910-45f0-8929-0d0bf2be6442") : object "kube-system"/"original-pull-secret" not registered
May 28 17:09:55.055894 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:55.055854    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hv7cn" event={"ID":"3089b2b0-3c18-4a11-b853-3cdb8fcf459a","Type":"ContainerStarted","Data":"289ecb7a08c0d131c1a08b8acc47c9101b85d93795c3608aefe6e40411955019"}
May 28 17:09:55.070154 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:55.070116    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hv7cn" podStartSLOduration=5.040335456 podStartE2EDuration="15.070101696s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.735193701 +0000 UTC m=+3.369106411" lastFinishedPulling="2026-05-28 17:09:52.764959938 +0000 UTC m=+13.398872651" observedRunningTime="2026-05-28 17:09:55.069517326 +0000 UTC m=+15.703430083" watchObservedRunningTime="2026-05-28 17:09:55.070101696 +0000 UTC m=+15.704014416"
May 28 17:09:55.966908 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:55.966874    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:55.967104 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:55.966879    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:55.967104 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:55.967004    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:55.967104 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:55.966879    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:55.967104 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:55.967094    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:09:55.967328 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:55.967174    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:56.653154 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:56.653113    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:56.653786 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:56.653670    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:57.597403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:57.597355    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:57.597568 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.597514    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:57.597628 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.597582    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:13.597561026 +0000 UTC m=+34.231473729 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:09:57.698677 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:57.698638    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:57.699296 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.698818    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:09:57.699296 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.698840    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:09:57.699296 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.698855    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:57.699296 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.698924    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:13.698903779 +0000 UTC m=+34.332816489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:09:57.967426 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:57.966892    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:57.967426 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.967011    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:09:57.967426 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:57.967096    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:57.967426 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.967235    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:09:57.967426 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:57.966892    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:57.967426 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:57.967335    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:58.520718 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:58.520637    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:58.520896 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:58.520755    2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 17:09:58.521518 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:58.521490    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dtzs2"
May 28 17:09:58.604979 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:58.604946    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:58.605332 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:58.605209    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:09:58.605332 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:58.605282    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret podName:741cdce6-f910-45f0-8929-0d0bf2be6442 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:06.605262822 +0000 UTC m=+27.239175528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret") pod "global-pull-secret-syncer-tc52b" (UID: "741cdce6-f910-45f0-8929-0d0bf2be6442") : object "kube-system"/"original-pull-secret" not registered
May 28 17:09:59.969896 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:59.969733    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:09:59.970359 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:59.970247    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:09:59.971369 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:09:59.970597    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:09:59.971369 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:59.970900    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:09:59.971369 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:59.971042    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:09:59.971592 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:09:59.971551    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:01.967334 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:01.967299    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:01.967334 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:01.967328    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:01.967926 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:01.967299    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:01.967926 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:01.967443    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:01.967926 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:01.967548    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:01.967926 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:01.967674    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:03.621446 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.620912    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 17:10:03.935931 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.935855    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T17:10:03.621299282Z","UUID":"7a182e1e-4d06-4de1-8326-4b455cacad16","Handler":null,"Name":"","Endpoint":""}
May 28 17:10:03.937209 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.937194    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 17:10:03.937297 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.937215    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 17:10:03.967115 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.967099    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:03.967191 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.967099    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:03.967241 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:03.967201    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:03.967333 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:03.967106    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:03.967471 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:03.967303    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:03.967471 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:03.967430    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:04.071901 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.071874    2574 generic.go:358] "Generic (PLEG): container finished" podID="4ba637e9-8b45-407d-8e68-50133573feac" containerID="1a6cd44c6e6925bcdaa76d36f6d4b5b28059a90345828848b1adeccc2fe74bd6" exitCode=0
May 28 17:10:04.072021 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.071953    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerDied","Data":"1a6cd44c6e6925bcdaa76d36f6d4b5b28059a90345828848b1adeccc2fe74bd6"}
May 28 17:10:04.075007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.074975    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"445d672f74b11c0b70d88134ab41c8f0a621d5823f0b1b1499ba4e5a858863aa"}
May 28 17:10:04.075103 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.075009    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"862f2743177d6463982b702a2f3417bcad6c8f1ce38126ad99837bacee3f1991"}
May 28 17:10:04.075103 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.075025    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"35f9b5e787b8e17826f814e8622a92e6c3a0f55a4618b5065df46bb560f352f2"}
May 28 17:10:04.075103 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.075043    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"3418ffbdf9a153af0aec004f2cd4e8881e10af5bc497d2946a1e5a5f400acffc"}
May 28 17:10:04.075103 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.075056    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"23c3bd766c9249d4dcf1bca1e738cb1d26b5b23bc5cc2f103aabd2f6a50e5c57"}
May 28 17:10:04.075103 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.075069    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"f70514434a9fda51977c00335b744a899d3f97ec1b06e586011ccb4e44b00c66"}
May 28 17:10:04.076244 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.076222    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d9w6x" event={"ID":"4e72d728-8f82-4fd9-b8cf-0377519b2328","Type":"ContainerStarted","Data":"8fccb52767eaa9021b0c811bc2c6e39e8ded11ff7cc3092660bd4398c0145bbb"}
May 28 17:10:04.077962 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.077935    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h" event={"ID":"860d8d43-a916-4d50-ac0c-7e0e927b00e3","Type":"ContainerStarted","Data":"841389da5432577b354f048fcc80ea54138473dab045c18dda6155c7458ed273"}
May 28 17:10:04.126366 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:04.126309    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-d9w6x" podStartSLOduration=3.438276144 podStartE2EDuration="24.126296977s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.70501981 +0000 UTC m=+3.338932514" lastFinishedPulling="2026-05-28 17:10:03.393040646 +0000 UTC m=+24.026953347" observedRunningTime="2026-05-28 17:10:04.12579304 +0000 UTC m=+24.759705761" watchObservedRunningTime="2026-05-28 17:10:04.126296977 +0000 UTC m=+24.760209697"
May 28 17:10:05.080958 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:05.080930    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h" event={"ID":"860d8d43-a916-4d50-ac0c-7e0e927b00e3","Type":"ContainerStarted","Data":"c6632ee8e9d8d5326a9367307e1a4dca2691885244ea3bbd8add6e918c19a32f"}
May 28 17:10:05.103302 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:05.103263    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5kx5h" podStartSLOduration=2.972704368 podStartE2EDuration="25.103252532s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.735038583 +0000 UTC m=+3.368951293" lastFinishedPulling="2026-05-28 17:10:04.865586754 +0000 UTC m=+25.499499457" observedRunningTime="2026-05-28 17:10:05.103119532 +0000 UTC m=+25.737032264" watchObservedRunningTime="2026-05-28 17:10:05.103252532 +0000 UTC m=+25.737165253"
May 28 17:10:05.967069 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:05.967035    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:05.967239 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:05.967035    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:05.967239 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:05.967044    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:05.967239 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:05.967214    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:05.967408 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:05.967294    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:05.967408 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:05.967124    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:06.084292 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:06.084256    2574 generic.go:358] "Generic (PLEG): container finished" podID="4ba637e9-8b45-407d-8e68-50133573feac" containerID="1d9363606baa9e9510b9063034cb243030b0e5902e307e7f96198b2ac4e88219" exitCode=0
May 28 17:10:06.084831 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:06.084351    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerDied","Data":"1d9363606baa9e9510b9063034cb243030b0e5902e307e7f96198b2ac4e88219"}
May 28 17:10:06.663950 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:06.663918    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:06.664099 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:06.664017    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 17:10:06.664099 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:06.664067    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret podName:741cdce6-f910-45f0-8929-0d0bf2be6442 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:22.664051843 +0000 UTC m=+43.297964542 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret") pod "global-pull-secret-syncer-tc52b" (UID: "741cdce6-f910-45f0-8929-0d0bf2be6442") : object "kube-system"/"original-pull-secret" not registered
May 28 17:10:07.088207 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:07.088181    2574 generic.go:358] "Generic (PLEG): container finished" podID="4ba637e9-8b45-407d-8e68-50133573feac" containerID="e39951426cdbbdd3005c8f2b42939f995835d4610da652fe3aa8f241ed648d25" exitCode=0
May 28 17:10:07.088567 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:07.088244    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerDied","Data":"e39951426cdbbdd3005c8f2b42939f995835d4610da652fe3aa8f241ed648d25"}
May 28 17:10:07.091303 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:07.091281    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"03787f2272823db324c0cf089b40a7c3bfdc57d91bf040ccb9f714660a2e7861"}
May 28 17:10:07.966480 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:07.966448    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:07.966653 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:07.966448    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:07.966653 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:07.966553    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:07.966762 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:07.966665    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:07.966762 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:07.966448    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:07.966762 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:07.966752    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:09.097857 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.097674    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" event={"ID":"98543425-257a-4fa1-b5ad-fbef8f1e40b6","Type":"ContainerStarted","Data":"4199ebca400e8206c3751bcefc2290375cf6b17090e3314fdea40fd2c6f56ddb"}
May 28 17:10:09.098306 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.098015    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:10:09.098306 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.098038    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:10:09.113049 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.113024    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:10:09.172753 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.172709    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5" podStartSLOduration=8.489175328 podStartE2EDuration="29.172697816s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.709390022 +0000 UTC m=+3.343302726" lastFinishedPulling="2026-05-28 17:10:03.3929125 +0000 UTC m=+24.026825214" observedRunningTime="2026-05-28 17:10:09.139436351 +0000 UTC m=+29.773349073" watchObservedRunningTime="2026-05-28 17:10:09.172697816 +0000 UTC m=+29.806610536"
May 28 17:10:09.967649 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.967614    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:09.967800 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.967697    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:09.967800 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:09.967734    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:09.967800 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:09.967763    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:09.967966 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:09.967807    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:09.967966 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:09.967908    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:10.100229 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.100150    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:10:10.116610 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.116583    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:10:10.266760 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.266663    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tc52b"]
May 28 17:10:10.266903 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.266774    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:10.266903 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:10.266870    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:10.269503 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.269480    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j9xtb"]
May 28 17:10:10.269627 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.269558    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:10.269688 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:10.269637    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:10.272750 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.272532    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdjr4"]
May 28 17:10:10.272750 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:10.272623    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:10.272750 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:10.272738    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:11.966949 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:11.966916    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:11.967426 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:11.966917    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:11.967426 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:11.967027    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:11.967426 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:11.967124    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:11.967426 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:11.966917    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:11.967426 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:11.967201    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:13.620392 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:13.620363    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:13.620736 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.620504    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:10:13.620736 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.620567    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:45.620553901 +0000 UTC m=+66.254466600 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 17:10:13.723815 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:13.721221    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:13.723815 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.721676    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 17:10:13.723815 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.721731    2574 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 17:10:13.723815 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.721750    2574 projected.go:194] Error preparing data for projected volume kube-api-access-2k7kb for pod openshift-network-diagnostics/network-check-target-sdjr4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:10:13.723815 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.721835    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb podName:e2083f78-5e30-4486-a9c7-cf59b8feebd1 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:45.721816636 +0000 UTC m=+66.355729361 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2k7kb" (UniqueName: "kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb") pod "network-check-target-sdjr4" (UID: "e2083f78-5e30-4486-a9c7-cf59b8feebd1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 17:10:13.966573 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:13.966541    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:13.966573 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:13.966568    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:13.966770 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:13.966634    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:13.966770 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.966645    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:13.966770 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.966726    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:13.966908 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:13.966790    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:14.109358 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:14.109268    2574 generic.go:358] "Generic (PLEG): container finished" podID="4ba637e9-8b45-407d-8e68-50133573feac" containerID="d829f9ec4aa14ec9422abae8d630a32c870ec9618503c2c71fedc9dc7d86c967" exitCode=0
May 28 17:10:14.109358 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:14.109326    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerDied","Data":"d829f9ec4aa14ec9422abae8d630a32c870ec9618503c2c71fedc9dc7d86c967"}
May 28 17:10:15.113468 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:15.113433    2574 generic.go:358] "Generic (PLEG): container finished" podID="4ba637e9-8b45-407d-8e68-50133573feac" containerID="8fca0c5113dde37524337e306ef20011590b9e343ac972d4d02c71410cbabf8d" exitCode=0
May 28 17:10:15.113852 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:15.113497    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerDied","Data":"8fca0c5113dde37524337e306ef20011590b9e343ac972d4d02c71410cbabf8d"}
May 28 17:10:15.966520 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:15.966485    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:15.966520 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:15.966502    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:15.966780 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:15.966491    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:15.966780 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:15.966614    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:10:15.966780 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:15.966717    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tc52b" podUID="741cdce6-f910-45f0-8929-0d0bf2be6442"
May 28 17:10:15.966780 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:15.966767    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdjr4" podUID="e2083f78-5e30-4486-a9c7-cf59b8feebd1"
May 28 17:10:16.120026 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:16.117930    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" event={"ID":"4ba637e9-8b45-407d-8e68-50133573feac","Type":"ContainerStarted","Data":"542f31a2ee6f70e8fb50d19e31a1659568841d6cc430ec37fd9e97982f530f94"}
May 28 17:10:16.146941 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:16.146895    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s9w8f" podStartSLOduration=5.340221292 podStartE2EDuration="36.1468812s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:09:42.731004808 +0000 UTC m=+3.364917523" lastFinishedPulling="2026-05-28 17:10:13.537664717 +0000 UTC m=+34.171577431" observedRunningTime="2026-05-28 17:10:16.146291367 +0000 UTC m=+36.780204088" watchObservedRunningTime="2026-05-28 17:10:16.1468812 +0000 UTC m=+36.780793951"
May 28 17:10:17.124061 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.123970    2574 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-141-155.ec2.internal" event="NodeReady"
May 28 17:10:17.124641 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.124104    2574 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 17:10:17.168427 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.168395    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-588bd6dc7f-6hpns"]
May 28 17:10:17.211474 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.209238    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.218659 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.214939    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 17:10:17.218659 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.215165    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 17:10:17.218659 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.215360    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 17:10:17.218659 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.215571    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zjnfc\""
May 28 17:10:17.219174 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.219148    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"]
May 28 17:10:17.222872 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.222811    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 17:10:17.242290 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.242267    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"]
May 28 17:10:17.242450 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.242431    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.245316 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.245296    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 28 17:10:17.245427 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.245296    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 28 17:10:17.245747 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.245730    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-f6jtj\""
May 28 17:10:17.265080 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.265060    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"]
May 28 17:10:17.265225 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.265209    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.267768 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.267747    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 17:10:17.267863 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.267747    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 17:10:17.268030 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.268012    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 17:10:17.268265 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.268250    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 17:10:17.268608 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.268587    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 17:10:17.268682 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.268614    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 17:10:17.268805 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.268791    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 17:10:17.281041 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.281024    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"]
May 28 17:10:17.281165 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.281151    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.283475 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.283460    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 17:10:17.298576 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.298560    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"]
May 28 17:10:17.298646 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.298581    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gzmmj"]
May 28 17:10:17.298698 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.298683    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.301392 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.301375    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2kclk\""
May 28 17:10:17.301491 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.301375    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 17:10:17.318921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.318903    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"]
May 28 17:10:17.319030 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.318927    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-588bd6dc7f-6hpns"]
May 28 17:10:17.319030 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.318940    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"]
May 28 17:10:17.319030 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.318950    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"]
May 28 17:10:17.319030 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.318963    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p7gwr"]
May 28 17:10:17.319030 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.319021    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.321384 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.321368    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 17:10:17.321637 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.321621    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmnj9\""
May 28 17:10:17.321693 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.321673    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 17:10:17.345057 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.345039    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gzmmj"]
May 28 17:10:17.345145 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.345061    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7gwr"]
May 28 17:10:17.345181 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.345164    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:17.348306 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348288    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jgwcl\""
May 28 17:10:17.348576 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348560    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 17:10:17.348642 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348576    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 17:10:17.348642 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348576    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-trusted-ca\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.348642 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348563    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 17:10:17.348773 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-registry-certificates\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.348773 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348670    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-installation-pull-secrets\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.348773 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348695    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-bound-sa-token\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.348773 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/138b31fa-fed6-4000-b655-54d63d677b41-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.348773 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348738    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-image-registry-private-configuration\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348818    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qxck\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-kube-api-access-6qxck\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348861    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-ca\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-hub\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.348963 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6b50cac-d931-4455-99aa-dce466196b5c-ca-trust-extracted\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.349184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.348986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7dd33fff-69cd-4898-a731-49486277ef41-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.349184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.349021    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.349184 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.349070    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42hwx\" (UniqueName: \"kubernetes.io/projected/138b31fa-fed6-4000-b655-54d63d677b41-kube-api-access-42hwx\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.450360 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450277    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.450360 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.450534 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450392    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6qxck\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-kube-api-access-6qxck\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.450534 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450453    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.450534 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450484    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-trusted-ca\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.450534 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450501    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/138b31fa-fed6-4000-b655-54d63d677b41-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.450534 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.450519    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:17.450721 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.450577    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:17.950557716 +0000 UTC m=+38.584470420 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:17.450721 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450524    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-config-volume\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.450721 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450673    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-installation-pull-secrets\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.450721 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450706    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-bound-sa-token\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.450921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450734    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-image-registry-private-configuration\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.450921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450772    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-ca\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.450921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d812356e-a597-4fa3-ab6e-62f84d7bf41c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74d758d978-p2rmn\" (UID: \"d812356e-a597-4fa3-ab6e-62f84d7bf41c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.450921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc9l9\" (UniqueName: \"kubernetes.io/projected/d812356e-a597-4fa3-ab6e-62f84d7bf41c-kube-api-access-qc9l9\") pod \"managed-serviceaccount-addon-agent-74d758d978-p2rmn\" (UID: \"d812356e-a597-4fa3-ab6e-62f84d7bf41c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.450921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450888    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-42hwx\" (UniqueName: \"kubernetes.io/projected/138b31fa-fed6-4000-b655-54d63d677b41-kube-api-access-42hwx\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fl86\" (UniqueName: \"kubernetes.io/projected/2ca10b38-5078-4ede-ae2a-5261f80affb6-kube-api-access-2fl86\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450958    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skv7x\" (UniqueName: \"kubernetes.io/projected/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-kube-api-access-skv7x\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.450992    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451030    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-hub\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451058    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6b50cac-d931-4455-99aa-dce466196b5c-ca-trust-extracted\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451086    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7dd33fff-69cd-4898-a731-49486277ef41-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-tmp-dir\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.451160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451148    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451180    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-registry-certificates\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451206    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451241    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-tmp\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451270    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451278    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/138b31fa-fed6-4000-b655-54d63d677b41-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451303    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff42q\" (UniqueName: \"kubernetes.io/projected/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-kube-api-access-ff42q\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.451531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451399    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6b50cac-d931-4455-99aa-dce466196b5c-ca-trust-extracted\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.451922 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451773    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-registry-certificates\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.451922 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.451853    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:17.451922 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.451865    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:17.451922 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.451917    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:17.951900761 +0000 UTC m=+38.585813460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:17.452122 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.451985    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-trusted-ca\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.452450 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.452429    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7dd33fff-69cd-4898-a731-49486277ef41-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.455975 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.455950    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-image-registry-private-configuration\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.456083 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.455980    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.456083 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.455995    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-hub\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.456083 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.455953    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-ca\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.456083 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.455955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-installation-pull-secrets\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.456083 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.456047    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/138b31fa-fed6-4000-b655-54d63d677b41-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.459117 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.459097    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-bound-sa-token\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.463512 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.463492    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qxck\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-kube-api-access-6qxck\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.463786 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.463765    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42hwx\" (UniqueName: \"kubernetes.io/projected/138b31fa-fed6-4000-b655-54d63d677b41-kube-api-access-42hwx\") pod \"cluster-proxy-proxy-agent-588b847b4b-pdkh9\" (UID: \"138b31fa-fed6-4000-b655-54d63d677b41\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.551727 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551684    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qc9l9\" (UniqueName: \"kubernetes.io/projected/d812356e-a597-4fa3-ab6e-62f84d7bf41c-kube-api-access-qc9l9\") pod \"managed-serviceaccount-addon-agent-74d758d978-p2rmn\" (UID: \"d812356e-a597-4fa3-ab6e-62f84d7bf41c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.551891 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551742    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2fl86\" (UniqueName: \"kubernetes.io/projected/2ca10b38-5078-4ede-ae2a-5261f80affb6-kube-api-access-2fl86\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:17.551891 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551770    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-skv7x\" (UniqueName: \"kubernetes.io/projected/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-kube-api-access-skv7x\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.551891 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551796    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:17.551891 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551829    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-tmp-dir\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.551891 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551861    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-tmp\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.552160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.551986    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ff42q\" (UniqueName: \"kubernetes.io/projected/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-kube-api-access-ff42q\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.552160 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.552026    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:17.552160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552029    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.552160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552057    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.552160 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.552082    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:18.052063426 +0000 UTC m=+38.685976129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:17.552160 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.552139    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:17.552441 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552174    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-config-volume\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.552441 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.552204    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:10:18.05218424 +0000 UTC m=+38.686096957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:17.552441 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552237    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-tmp-dir\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.552441 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552246    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d812356e-a597-4fa3-ab6e-62f84d7bf41c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74d758d978-p2rmn\" (UID: \"d812356e-a597-4fa3-ab6e-62f84d7bf41c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.552441 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552273    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-tmp\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.552605 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.552593    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-config-volume\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.554433 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.554413    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-klusterlet-config\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.554539 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.554470    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d812356e-a597-4fa3-ab6e-62f84d7bf41c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74d758d978-p2rmn\" (UID: \"d812356e-a597-4fa3-ab6e-62f84d7bf41c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.571401 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.571382    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skv7x\" (UniqueName: \"kubernetes.io/projected/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-kube-api-access-skv7x\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:17.571663 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.571644    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff42q\" (UniqueName: \"kubernetes.io/projected/77de7643-e86a-4a32-9f6c-ed8b9c9c9371-kube-api-access-ff42q\") pod \"klusterlet-addon-workmgr-6d845845bb-l2xn6\" (UID: \"77de7643-e86a-4a32-9f6c-ed8b9c9c9371\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.571730 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.571716    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fl86\" (UniqueName: \"kubernetes.io/projected/2ca10b38-5078-4ede-ae2a-5261f80affb6-kube-api-access-2fl86\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:17.572158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.572143    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc9l9\" (UniqueName: \"kubernetes.io/projected/d812356e-a597-4fa3-ab6e-62f84d7bf41c-kube-api-access-qc9l9\") pod \"managed-serviceaccount-addon-agent-74d758d978-p2rmn\" (UID: \"d812356e-a597-4fa3-ab6e-62f84d7bf41c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.587130 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.587113    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:10:17.605863 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.605839    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:17.605965 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.605869    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"
May 28 17:10:17.760145 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.759984    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn"]
May 28 17:10:17.762942 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:10:17.762914    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd812356e_a597_4fa3_ab6e_62f84d7bf41c.slice/crio-4933712c4d84a5aeb25a1466c40fb327f586d9a64c0aa9700e027d0b61d755b6 WatchSource:0}: Error finding container 4933712c4d84a5aeb25a1466c40fb327f586d9a64c0aa9700e027d0b61d755b6: Status 404 returned error can't find the container with id 4933712c4d84a5aeb25a1466c40fb327f586d9a64c0aa9700e027d0b61d755b6
May 28 17:10:17.771410 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.768646    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"]
May 28 17:10:17.772431 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:10:17.772405    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77de7643_e86a_4a32_9f6c_ed8b9c9c9371.slice/crio-bfa8669a7ea6aac2b0ee0eb4eac062e3db7aec204c53de71f524e88207da1bd2 WatchSource:0}: Error finding container bfa8669a7ea6aac2b0ee0eb4eac062e3db7aec204c53de71f524e88207da1bd2: Status 404 returned error can't find the container with id bfa8669a7ea6aac2b0ee0eb4eac062e3db7aec204c53de71f524e88207da1bd2
May 28 17:10:17.772851 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.772829    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"]
May 28 17:10:17.775279 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:10:17.775262    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod138b31fa_fed6_4000_b655_54d63d677b41.slice/crio-b9703e3250340245669177c0ebd3bd34cf3b8791e6e43d6f34b6012fa32ba3a4 WatchSource:0}: Error finding container b9703e3250340245669177c0ebd3bd34cf3b8791e6e43d6f34b6012fa32ba3a4: Status 404 returned error can't find the container with id b9703e3250340245669177c0ebd3bd34cf3b8791e6e43d6f34b6012fa32ba3a4
May 28 17:10:17.956110 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.956068    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:17.956284 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.956140    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:17.956284 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.956230    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:17.956429 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.956312    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:18.956296162 +0000 UTC m=+39.590208869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:17.956429 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.956237    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:17.956429 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.956332    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:17.956429 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:17.956382    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:18.956374213 +0000 UTC m=+39.590286916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:17.966884 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.966761    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:17.966884 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.966777    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:17.966884 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.966763    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:17.969486 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.969464    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 17:10:17.969739 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.969723    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 17:10:17.969861 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.969748    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 17:10:17.969861 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.969726    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hshhh\""
May 28 17:10:17.969861 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.969726    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 17:10:17.970092 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:17.970079    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t7ndh\""
May 28 17:10:18.057473 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.057445    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:18.057629 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.057508    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:18.057629 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.057612    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:18.057697 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.057656    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:18.057697 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.057676    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:19.057663083 +0000 UTC m=+39.691575786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:18.057759 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.057707    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:10:19.057694399 +0000 UTC m=+39.691607097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:18.122927 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.122893    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6" event={"ID":"77de7643-e86a-4a32-9f6c-ed8b9c9c9371","Type":"ContainerStarted","Data":"bfa8669a7ea6aac2b0ee0eb4eac062e3db7aec204c53de71f524e88207da1bd2"}
May 28 17:10:18.123911 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.123881    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" event={"ID":"138b31fa-fed6-4000-b655-54d63d677b41","Type":"ContainerStarted","Data":"b9703e3250340245669177c0ebd3bd34cf3b8791e6e43d6f34b6012fa32ba3a4"}
May 28 17:10:18.124790 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.124764    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn" event={"ID":"d812356e-a597-4fa3-ab6e-62f84d7bf41c","Type":"ContainerStarted","Data":"4933712c4d84a5aeb25a1466c40fb327f586d9a64c0aa9700e027d0b61d755b6"}
May 28 17:10:18.964888 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.964846    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:18.965075 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:18.964981    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:18.965158 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.965141    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:18.965223 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.965164    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:18.965277 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.965223    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:20.965204384 +0000 UTC m=+41.599117086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:18.965277 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.965139    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:18.965410 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:18.965298    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:20.965277735 +0000 UTC m=+41.599190458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:19.066484 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:19.066448    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:19.066673 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:19.066584    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:19.066732 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:19.066713    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:19.066790 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:19.066772    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:21.066754516 +0000 UTC m=+41.700667224 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:19.067195 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:19.067172    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:19.067310 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:19.067234    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:10:21.067217898 +0000 UTC m=+41.701130613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:20.984332 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:20.984286    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:20.984791 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:20.984398    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:20.984791 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:20.984454    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:20.984791 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:20.984471    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:20.984791 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:20.984532    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:24.984515711 +0000 UTC m=+45.618428430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:20.984791 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:20.984571    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:20.984791 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:20.984647    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:24.984628227 +0000 UTC m=+45.618540930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:21.085478 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:21.085442    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:21.085633 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:21.085531    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:21.085633 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:21.085596    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:21.085759 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:21.085650    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:21.085759 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:21.085677    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:25.085656994 +0000 UTC m=+45.719569713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:21.085759 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:21.085699    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:10:25.08568833 +0000 UTC m=+45.719601029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:22.699106 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:22.699069    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:22.702649 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:22.702619    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/741cdce6-f910-45f0-8929-0d0bf2be6442-original-pull-secret\") pod \"global-pull-secret-syncer-tc52b\" (UID: \"741cdce6-f910-45f0-8929-0d0bf2be6442\") " pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:22.788878 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:22.788846    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tc52b"
May 28 17:10:25.017350 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:25.017270    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:25.017669 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:25.017333    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:25.017669 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.017441    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:25.017669 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.017457    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:25.017669 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.017510    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:33.017497056 +0000 UTC m=+53.651409755 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:25.017669 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.017459    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:25.017669 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.017587    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:33.017573849 +0000 UTC m=+53.651486554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:25.118149 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:25.118116    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:25.118276 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:25.118171    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:25.118276 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.118259    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:25.118276 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.118260    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:25.118435 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.118306    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:10:33.118293422 +0000 UTC m=+53.752206126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:25.118435 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:25.118318    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:33.118312279 +0000 UTC m=+53.752224978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:25.369732 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:25.369658    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tc52b"]
May 28 17:10:25.372509 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:10:25.372484    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741cdce6_f910_45f0_8929_0d0bf2be6442.slice/crio-c73d5eb54450a42b461260a5617bfe0370d289bfc37958e18e3f685bb7205e3f WatchSource:0}: Error finding container c73d5eb54450a42b461260a5617bfe0370d289bfc37958e18e3f685bb7205e3f: Status 404 returned error can't find the container with id c73d5eb54450a42b461260a5617bfe0370d289bfc37958e18e3f685bb7205e3f
May 28 17:10:26.142269 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.142236    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" event={"ID":"138b31fa-fed6-4000-b655-54d63d677b41","Type":"ContainerStarted","Data":"d5ff14ac255b070590bce6171a77734e88c8a2aec04b95d776ac81390358679d"}
May 28 17:10:26.143282 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.143258    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tc52b" event={"ID":"741cdce6-f910-45f0-8929-0d0bf2be6442","Type":"ContainerStarted","Data":"c73d5eb54450a42b461260a5617bfe0370d289bfc37958e18e3f685bb7205e3f"}
May 28 17:10:26.144468 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.144447    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn" event={"ID":"d812356e-a597-4fa3-ab6e-62f84d7bf41c","Type":"ContainerStarted","Data":"9dd5c141032e5fbdfa4bf2a34e4e0c5985df2b6a0c2fc627a6f59b725250921c"}
May 28 17:10:26.148863 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.148839    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6" event={"ID":"77de7643-e86a-4a32-9f6c-ed8b9c9c9371","Type":"ContainerStarted","Data":"83fd1273458705d4ea049cfa35fa988388870afad5cefcadcb222fddac1de9cc"}
May 28 17:10:26.149079 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.149067    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:26.150744 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.150723    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:10:26.169359 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.169301    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn" podStartSLOduration=37.696366122 podStartE2EDuration="45.169285142s" podCreationTimestamp="2026-05-28 17:09:41 +0000 UTC" firstStartedPulling="2026-05-28 17:10:17.76475831 +0000 UTC m=+38.398671009" lastFinishedPulling="2026-05-28 17:10:25.237677323 +0000 UTC m=+45.871590029" observedRunningTime="2026-05-28 17:10:26.167354601 +0000 UTC m=+46.801267324" watchObservedRunningTime="2026-05-28 17:10:26.169285142 +0000 UTC m=+46.803197865"
May 28 17:10:26.196456 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:26.196404    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6" podStartSLOduration=37.730836033 podStartE2EDuration="45.196391764s" podCreationTimestamp="2026-05-28 17:09:41 +0000 UTC" firstStartedPulling="2026-05-28 17:10:17.773676347 +0000 UTC m=+38.407589058" lastFinishedPulling="2026-05-28 17:10:25.23923209 +0000 UTC m=+45.873144789" observedRunningTime="2026-05-28 17:10:26.194763853 +0000 UTC m=+46.828676575" watchObservedRunningTime="2026-05-28 17:10:26.196391764 +0000 UTC m=+46.830304488"
May 28 17:10:30.158144 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:30.158057    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tc52b" event={"ID":"741cdce6-f910-45f0-8929-0d0bf2be6442","Type":"ContainerStarted","Data":"346d160679af3b78d57f713a29ebdb0072f56e08d1f30ff78bb65b9688af50fc"}
May 28 17:10:30.183865 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:30.183817    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tc52b" podStartSLOduration=35.768827876 podStartE2EDuration="40.183803052s" podCreationTimestamp="2026-05-28 17:09:50 +0000 UTC" firstStartedPulling="2026-05-28 17:10:25.374858807 +0000 UTC m=+46.008771520" lastFinishedPulling="2026-05-28 17:10:29.789833996 +0000 UTC m=+50.423746696" observedRunningTime="2026-05-28 17:10:30.183018252 +0000 UTC m=+50.816930974" watchObservedRunningTime="2026-05-28 17:10:30.183803052 +0000 UTC m=+50.817715772"
May 28 17:10:33.077717 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.077674    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:33.078130 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.077778    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:33.078130 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.077806    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:33.078130 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.077873    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:49.077857473 +0000 UTC m=+69.711770179 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:33.078130 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.077904    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:33.078130 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.077922    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:33.078130 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.077966    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:10:49.077953774 +0000 UTC m=+69.711866474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:33.166657 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.166613    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" event={"ID":"138b31fa-fed6-4000-b655-54d63d677b41","Type":"ContainerStarted","Data":"855fd36470476ca7e51f5a79ad81eceb093de62cd75bcab80c2dad195f51b4c1"}
May 28 17:10:33.166657 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.166649    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" event={"ID":"138b31fa-fed6-4000-b655-54d63d677b41","Type":"ContainerStarted","Data":"98d019387b9f10cf84b1712ee43f6dfc9cce189e79294a27c1bc7f4550e0ef9c"}
May 28 17:10:33.178453 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.178427    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:33.178599 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.178525    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:33.178599 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.178566    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:33.178711 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.178626    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:10:49.178607237 +0000 UTC m=+69.812519939 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:33.178711 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.178652    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:33.178815 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:33.178719    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:10:49.178701301 +0000 UTC m=+69.812614009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:33.208097 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:33.208055    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" podStartSLOduration=37.656230629 podStartE2EDuration="52.208041628s" podCreationTimestamp="2026-05-28 17:09:41 +0000 UTC" firstStartedPulling="2026-05-28 17:10:17.77689847 +0000 UTC m=+38.410811172" lastFinishedPulling="2026-05-28 17:10:32.328709473 +0000 UTC m=+52.962622171" observedRunningTime="2026-05-28 17:10:33.20752926 +0000 UTC m=+53.841441981" watchObservedRunningTime="2026-05-28 17:10:33.208041628 +0000 UTC m=+53.841954378"
May 28 17:10:42.114073 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:42.114034    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxzl5"
May 28 17:10:45.670139 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.670093    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:10:45.673236 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.673218    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 17:10:45.680670 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:45.680656    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 17:10:45.680738 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:45.680720    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:11:49.68070184 +0000 UTC m=+130.314614562 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : secret "metrics-daemon-secret" not found
May 28 17:10:45.770637 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.770605    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:45.774551 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.774533    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 17:10:45.784444 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.784427    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 17:10:45.794834 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.794810    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k7kb\" (UniqueName: \"kubernetes.io/projected/e2083f78-5e30-4486-a9c7-cf59b8feebd1-kube-api-access-2k7kb\") pod \"network-check-target-sdjr4\" (UID: \"e2083f78-5e30-4486-a9c7-cf59b8feebd1\") " pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:45.880567 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.880533    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hshhh\""
May 28 17:10:45.888623 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:45.888607    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:46.009873 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:46.009845    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdjr4"]
May 28 17:10:46.012563 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:10:46.012534    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2083f78_5e30_4486_a9c7_cf59b8feebd1.slice/crio-0383cc456393319c2df20ee9b61917f8c5b2e9ef3197e5f06f585b80a754c2f8 WatchSource:0}: Error finding container 0383cc456393319c2df20ee9b61917f8c5b2e9ef3197e5f06f585b80a754c2f8: Status 404 returned error can't find the container with id 0383cc456393319c2df20ee9b61917f8c5b2e9ef3197e5f06f585b80a754c2f8
May 28 17:10:46.199768 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:46.199700    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdjr4" event={"ID":"e2083f78-5e30-4486-a9c7-cf59b8feebd1","Type":"ContainerStarted","Data":"0383cc456393319c2df20ee9b61917f8c5b2e9ef3197e5f06f585b80a754c2f8"}
May 28 17:10:49.095206 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.095179    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:10:49.095508 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.095261    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:10:49.095508 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.095319    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:10:49.095508 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.095361    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:10:49.095508 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.095371    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:10:49.095508 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.095401    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:11:21.095384845 +0000 UTC m=+101.729297559 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:10:49.095508 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.095419    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:11:21.095410373 +0000 UTC m=+101.729323071 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:10:49.196128 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.196099    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:10:49.196256 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.196186    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:10:49.196256 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.196239    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:10:49.196324 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.196283    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:10:49.196324 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.196305    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:11:21.196281028 +0000 UTC m=+101.830193727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:10:49.196324 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:10:49.196322    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:11:21.196310364 +0000 UTC m=+101.830223064 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:10:49.208172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.208145    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdjr4" event={"ID":"e2083f78-5e30-4486-a9c7-cf59b8feebd1","Type":"ContainerStarted","Data":"bcc65eee665484c5b2bef452593a9414fc339462c8a8ef5a87ea1caea5936b0f"}
May 28 17:10:49.208297 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.208279    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:10:49.225782 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:10:49.225745    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sdjr4" podStartSLOduration=66.244370855 podStartE2EDuration="1m9.225732754s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:10:46.014396543 +0000 UTC m=+66.648309242" lastFinishedPulling="2026-05-28 17:10:48.995758428 +0000 UTC m=+69.629671141" observedRunningTime="2026-05-28 17:10:49.22538298 +0000 UTC m=+69.859295704" watchObservedRunningTime="2026-05-28 17:10:49.225732754 +0000 UTC m=+69.859645466"
May 28 17:11:20.212207 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:11:20.212127    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sdjr4"
May 28 17:11:21.137982 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:11:21.137942    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:11:21.138167 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.138096    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:11:21.138217 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:11:21.138159    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:11:21.138217 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.138182    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:25.138162276 +0000 UTC m=+165.772074989 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:11:21.138321 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.138239    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:11:21.138321 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.138250    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:11:21.138321 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.138283    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:12:25.138274804 +0000 UTC m=+165.772187502 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:11:21.239303 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:11:21.239265    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:11:21.239748 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:11:21.239355    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:11:21.239748 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.239444    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:11:21.239748 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.239453    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:11:21.239748 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.239507    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:12:25.239491022 +0000 UTC m=+165.873403727 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:11:21.239748 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:21.239520    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:12:25.239514236 +0000 UTC m=+165.873426935 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:11:49.749494 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:11:49.749445    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:11:49.749968 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:49.749609    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 17:11:49.749968 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:11:49.749722    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs podName:3a5cb798-cc5b-4fdf-9671-67201a9e7718 nodeName:}" failed. No retries permitted until 2026-05-28 17:13:51.749704193 +0000 UTC m=+252.383616913 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs") pod "network-metrics-daemon-j9xtb" (UID: "3a5cb798-cc5b-4fdf-9671-67201a9e7718") : secret "metrics-daemon-secret" not found
May 28 17:12:20.225610 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:20.225518    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns" podUID="a6b50cac-d931-4455-99aa-dce466196b5c"
May 28 17:12:20.250731 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:20.250709    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz" podUID="7dd33fff-69cd-4898-a731-49486277ef41"
May 28 17:12:20.340430 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:20.340384    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gzmmj" podUID="8dd0cee6-4a7d-46d3-9b45-1a2679a302ad"
May 28 17:12:20.352603 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:20.352585    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-p7gwr" podUID="2ca10b38-5078-4ede-ae2a-5261f80affb6"
May 28 17:12:20.418075 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:20.418054    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:12:20.418188 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:20.418056    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gzmmj"
May 28 17:12:20.418226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:20.418059    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:12:20.983601 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:20.983559    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-j9xtb" podUID="3a5cb798-cc5b-4fdf-9671-67201a9e7718"
May 28 17:12:25.204165 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.204127    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") pod \"image-registry-588bd6dc7f-6hpns\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") " pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:12:25.204677 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.204183    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:12:25.204677 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.204280    2574 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 17:12:25.204677 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.204299    2574 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-588bd6dc7f-6hpns: secret "image-registry-tls" not found
May 28 17:12:25.204677 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.204364    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls podName:a6b50cac-d931-4455-99aa-dce466196b5c nodeName:}" failed. No retries permitted until 2026-05-28 17:14:27.204333118 +0000 UTC m=+287.838245821 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls") pod "image-registry-588bd6dc7f-6hpns" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c") : secret "image-registry-tls" not found
May 28 17:12:25.204677 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.204288    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
May 28 17:12:25.204677 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.204393    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert podName:7dd33fff-69cd-4898-a731-49486277ef41 nodeName:}" failed. No retries permitted until 2026-05-28 17:14:27.204387021 +0000 UTC m=+287.838299720 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert") pod "networking-console-plugin-d4d8f94f9-8rlrz" (UID: "7dd33fff-69cd-4898-a731-49486277ef41") : secret "networking-console-plugin-cert" not found
May 28 17:12:25.305191 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.305161    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:12:25.305375 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.305247    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:12:25.305375 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.305288    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 17:12:25.305375 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.305351    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls podName:8dd0cee6-4a7d-46d3-9b45-1a2679a302ad nodeName:}" failed. No retries permitted until 2026-05-28 17:14:27.305322661 +0000 UTC m=+287.939235360 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls") pod "dns-default-gzmmj" (UID: "8dd0cee6-4a7d-46d3-9b45-1a2679a302ad") : secret "dns-default-metrics-tls" not found
May 28 17:12:25.305375 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.305372    2574 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 17:12:25.305563 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:12:25.305436    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert podName:2ca10b38-5078-4ede-ae2a-5261f80affb6 nodeName:}" failed. No retries permitted until 2026-05-28 17:14:27.305420762 +0000 UTC m=+287.939333463 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert") pod "ingress-canary-p7gwr" (UID: "2ca10b38-5078-4ede-ae2a-5261f80affb6") : secret "canary-serving-cert" not found
May 28 17:12:25.429958 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.429920    2574 generic.go:358] "Generic (PLEG): container finished" podID="d812356e-a597-4fa3-ab6e-62f84d7bf41c" containerID="9dd5c141032e5fbdfa4bf2a34e4e0c5985df2b6a0c2fc627a6f59b725250921c" exitCode=255
May 28 17:12:25.430058 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.429995    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn" event={"ID":"d812356e-a597-4fa3-ab6e-62f84d7bf41c","Type":"ContainerDied","Data":"9dd5c141032e5fbdfa4bf2a34e4e0c5985df2b6a0c2fc627a6f59b725250921c"}
May 28 17:12:25.430335 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.430318    2574 scope.go:117] "RemoveContainer" containerID="9dd5c141032e5fbdfa4bf2a34e4e0c5985df2b6a0c2fc627a6f59b725250921c"
May 28 17:12:25.431209 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.431190    2574 generic.go:358] "Generic (PLEG): container finished" podID="77de7643-e86a-4a32-9f6c-ed8b9c9c9371" containerID="83fd1273458705d4ea049cfa35fa988388870afad5cefcadcb222fddac1de9cc" exitCode=1
May 28 17:12:25.431292 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.431225    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6" event={"ID":"77de7643-e86a-4a32-9f6c-ed8b9c9c9371","Type":"ContainerDied","Data":"83fd1273458705d4ea049cfa35fa988388870afad5cefcadcb222fddac1de9cc"}
May 28 17:12:25.431607 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:25.431532    2574 scope.go:117] "RemoveContainer" containerID="83fd1273458705d4ea049cfa35fa988388870afad5cefcadcb222fddac1de9cc"
May 28 17:12:26.149297 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:26.149259    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:12:26.434781 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:26.434696    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74d758d978-p2rmn" event={"ID":"d812356e-a597-4fa3-ab6e-62f84d7bf41c","Type":"ContainerStarted","Data":"7861f48bcb931c2eb6e0ea0c08291c70eb0b71782ee65b5400da0b8365b56a05"}
May 28 17:12:26.436092 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:26.436066    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6" event={"ID":"77de7643-e86a-4a32-9f6c-ed8b9c9c9371","Type":"ContainerStarted","Data":"9c610c312d53447ec142fa2e69430a347e2d5f27ab285b30008784ea163c9b16"}
May 28 17:12:26.436278 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:26.436262    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:12:26.436809 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:26.436784    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6d845845bb-l2xn6"
May 28 17:12:28.220518 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:28.220489    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bvsbc_fd0fc226-3bdc-4e62-b9d4-0ccadcc58379/dns-node-resolver/0.log"
May 28 17:12:28.819301 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:28.819274    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7mc88_a8cc9d5d-aa7b-42f4-aa40-11579b736fa8/node-ca/0.log"
May 28 17:12:31.967007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:31.966969    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:12:32.967356 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:12:32.967313    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:13:00.927161 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.927124    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-slrlm"]
May 28 17:13:00.930148 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.930134    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:00.932831 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.932807    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wlzz5\""
May 28 17:13:00.932946 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.932832    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 17:13:00.933670 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.933649    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 17:13:00.933790 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.933702    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 17:13:00.933790 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.933653    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 17:13:00.944430 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:00.944408    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-slrlm"]
May 28 17:13:01.073251 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.073228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b76ca09e-5776-4b63-91ba-687aaf80972b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.073403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.073276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b76ca09e-5776-4b63-91ba-687aaf80972b-crio-socket\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.073403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.073308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b76ca09e-5776-4b63-91ba-687aaf80972b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.073403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.073327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76ca09e-5776-4b63-91ba-687aaf80972b-data-volume\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.073403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.073373    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2p2j\" (UniqueName: \"kubernetes.io/projected/b76ca09e-5776-4b63-91ba-687aaf80972b-kube-api-access-r2p2j\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.174650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.174622    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r2p2j\" (UniqueName: \"kubernetes.io/projected/b76ca09e-5776-4b63-91ba-687aaf80972b-kube-api-access-r2p2j\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.174777 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.174678    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b76ca09e-5776-4b63-91ba-687aaf80972b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.174858 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.174841    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b76ca09e-5776-4b63-91ba-687aaf80972b-crio-socket\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.174911 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.174896    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b76ca09e-5776-4b63-91ba-687aaf80972b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.174969 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.174927    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76ca09e-5776-4b63-91ba-687aaf80972b-data-volume\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.175015 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.174960    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b76ca09e-5776-4b63-91ba-687aaf80972b-crio-socket\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.175183 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.175168    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b76ca09e-5776-4b63-91ba-687aaf80972b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.175217 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.175175    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b76ca09e-5776-4b63-91ba-687aaf80972b-data-volume\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.177199 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.177140    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b76ca09e-5776-4b63-91ba-687aaf80972b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.184987 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.184967    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2p2j\" (UniqueName: \"kubernetes.io/projected/b76ca09e-5776-4b63-91ba-687aaf80972b-kube-api-access-r2p2j\") pod \"insights-runtime-extractor-slrlm\" (UID: \"b76ca09e-5776-4b63-91ba-687aaf80972b\") " pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.238660 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.238638    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-slrlm"
May 28 17:13:01.355834 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.355803    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-slrlm"]
May 28 17:13:01.358823 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:13:01.358798    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76ca09e_5776_4b63_91ba_687aaf80972b.slice/crio-695b4a57af9bca02d4ae454d42f044b59177abd7ecbdf46c5f7d3ec787cfc66a WatchSource:0}: Error finding container 695b4a57af9bca02d4ae454d42f044b59177abd7ecbdf46c5f7d3ec787cfc66a: Status 404 returned error can't find the container with id 695b4a57af9bca02d4ae454d42f044b59177abd7ecbdf46c5f7d3ec787cfc66a
May 28 17:13:01.519545 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.519467    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slrlm" event={"ID":"b76ca09e-5776-4b63-91ba-687aaf80972b","Type":"ContainerStarted","Data":"e8e0b53c9386205d2bb2da5a67df086466aa6f202ab03f245aaa2f248349d6b5"}
May 28 17:13:01.519545 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:01.519505    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slrlm" event={"ID":"b76ca09e-5776-4b63-91ba-687aaf80972b","Type":"ContainerStarted","Data":"695b4a57af9bca02d4ae454d42f044b59177abd7ecbdf46c5f7d3ec787cfc66a"}
May 28 17:13:02.523076 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:02.523005    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slrlm" event={"ID":"b76ca09e-5776-4b63-91ba-687aaf80972b","Type":"ContainerStarted","Data":"5561740b091b958dc7c575626c4972e9a026bacde3968a9b731127001bd5a883"}
May 28 17:13:04.532203 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:04.532169    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-slrlm" event={"ID":"b76ca09e-5776-4b63-91ba-687aaf80972b","Type":"ContainerStarted","Data":"9afb770386b9e977e5860b5e0e50a873f369795a8d8151b3869561e0bb7d75d3"}
May 28 17:13:04.549650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:04.549602    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-slrlm" podStartSLOduration=2.047114452 podStartE2EDuration="4.549587282s" podCreationTimestamp="2026-05-28 17:13:00 +0000 UTC" firstStartedPulling="2026-05-28 17:13:01.415678988 +0000 UTC m=+202.049591691" lastFinishedPulling="2026-05-28 17:13:03.918151823 +0000 UTC m=+204.552064521" observedRunningTime="2026-05-28 17:13:04.548761261 +0000 UTC m=+205.182673981" watchObservedRunningTime="2026-05-28 17:13:04.549587282 +0000 UTC m=+205.183500002"
May 28 17:13:07.588763 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:07.588719    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" podUID="138b31fa-fed6-4000-b655-54d63d677b41" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:13:13.194636 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.194603    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gr8n4"]
May 28 17:13:13.197774 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.197758    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.200377 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.200356    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 17:13:13.200500 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.200484    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 17:13:13.200555 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.200542    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 17:13:13.200638 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.200625    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 17:13:13.201253 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.201236    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 17:13:13.201397 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.201384    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 17:13:13.201457 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.201424    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8795h\""
May 28 17:13:13.366038 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.365992    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-root\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366038 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mzwv\" (UniqueName: \"kubernetes.io/projected/0045c394-d007-41ed-9cee-38f5a1a9d4fd-kube-api-access-4mzwv\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0045c394-d007-41ed-9cee-38f5a1a9d4fd-metrics-client-ca\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-textfile\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-tls\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366370 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366370 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366262    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-sys\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.366370 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.366279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-wtmp\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467306 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467233    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-root\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467306 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467269    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4mzwv\" (UniqueName: \"kubernetes.io/projected/0045c394-d007-41ed-9cee-38f5a1a9d4fd-kube-api-access-4mzwv\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467496 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467310    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0045c394-d007-41ed-9cee-38f5a1a9d4fd-metrics-client-ca\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467496 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467353    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-root\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467496 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467361    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-textfile\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467496 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467411    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467496 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467432    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-tls\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467496 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467469    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467766 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467512    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-sys\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467766 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467541    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-wtmp\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467766 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467631    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-textfile\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467766 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467639    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-sys\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.467766 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.467701    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-wtmp\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.468033 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.468016    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0045c394-d007-41ed-9cee-38f5a1a9d4fd-metrics-client-ca\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.468114 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.468072    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-accelerators-collector-config\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.469650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.469633    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.469740 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.469706    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0045c394-d007-41ed-9cee-38f5a1a9d4fd-node-exporter-tls\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.478597 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.478575    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mzwv\" (UniqueName: \"kubernetes.io/projected/0045c394-d007-41ed-9cee-38f5a1a9d4fd-kube-api-access-4mzwv\") pod \"node-exporter-gr8n4\" (UID: \"0045c394-d007-41ed-9cee-38f5a1a9d4fd\") " pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.506490 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.506469    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gr8n4"
May 28 17:13:13.514412 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:13:13.514331    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0045c394_d007_41ed_9cee_38f5a1a9d4fd.slice/crio-f91ddd74fb7f826fb6cbc46fa0e36d82a835b755152aa09977bf625457c6fe5a WatchSource:0}: Error finding container f91ddd74fb7f826fb6cbc46fa0e36d82a835b755152aa09977bf625457c6fe5a: Status 404 returned error can't find the container with id f91ddd74fb7f826fb6cbc46fa0e36d82a835b755152aa09977bf625457c6fe5a
May 28 17:13:13.553378 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:13.553336    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gr8n4" event={"ID":"0045c394-d007-41ed-9cee-38f5a1a9d4fd","Type":"ContainerStarted","Data":"f91ddd74fb7f826fb6cbc46fa0e36d82a835b755152aa09977bf625457c6fe5a"}
May 28 17:13:14.558462 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:14.558433    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gr8n4" event={"ID":"0045c394-d007-41ed-9cee-38f5a1a9d4fd","Type":"ContainerStarted","Data":"97f7422ab097049c090181723355fd126cfaec33237311d04486ad0ca4e14f98"}
May 28 17:13:15.562314 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:15.562278    2574 generic.go:358] "Generic (PLEG): container finished" podID="0045c394-d007-41ed-9cee-38f5a1a9d4fd" containerID="97f7422ab097049c090181723355fd126cfaec33237311d04486ad0ca4e14f98" exitCode=0
May 28 17:13:15.562696 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:15.562369    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gr8n4" event={"ID":"0045c394-d007-41ed-9cee-38f5a1a9d4fd","Type":"ContainerDied","Data":"97f7422ab097049c090181723355fd126cfaec33237311d04486ad0ca4e14f98"}
May 28 17:13:16.570690 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:16.570655    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gr8n4" event={"ID":"0045c394-d007-41ed-9cee-38f5a1a9d4fd","Type":"ContainerStarted","Data":"c8f0c64fe893ddead3cdadd5b31846621a9abfd5c35eda467404051503290ec6"}
May 28 17:13:16.570690 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:16.570690    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gr8n4" event={"ID":"0045c394-d007-41ed-9cee-38f5a1a9d4fd","Type":"ContainerStarted","Data":"8660c154f5318f03e7ace0ff6ee398908ce92426021499b2efe5482126ee1a73"}
May 28 17:13:16.593481 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:16.593436    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gr8n4" podStartSLOduration=2.6328678979999998 podStartE2EDuration="3.59342165s" podCreationTimestamp="2026-05-28 17:13:13 +0000 UTC" firstStartedPulling="2026-05-28 17:13:13.516327325 +0000 UTC m=+214.150240024" lastFinishedPulling="2026-05-28 17:13:14.476881077 +0000 UTC m=+215.110793776" observedRunningTime="2026-05-28 17:13:16.593243542 +0000 UTC m=+217.227156285" watchObservedRunningTime="2026-05-28 17:13:16.59342165 +0000 UTC m=+217.227334372"
May 28 17:13:17.588985 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:17.588951    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" podUID="138b31fa-fed6-4000-b655-54d63d677b41" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:13:23.327913 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.327882    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-588bd6dc7f-6hpns"]
May 28 17:13:23.328270 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:13:23.328073    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns" podUID="a6b50cac-d931-4455-99aa-dce466196b5c"
May 28 17:13:23.584861 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.584789    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:13:23.588722 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.588704    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:13:23.646883 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.646864    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-bound-sa-token\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.646939 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.646897    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-trusted-ca\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.646939 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.646929    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-installation-pull-secrets\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.647007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.646966    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qxck\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-kube-api-access-6qxck\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.647007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.646989    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-registry-certificates\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.647074 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.647031    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-image-registry-private-configuration\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.647074 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.647055    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6b50cac-d931-4455-99aa-dce466196b5c-ca-trust-extracted\") pod \"a6b50cac-d931-4455-99aa-dce466196b5c\" (UID: \"a6b50cac-d931-4455-99aa-dce466196b5c\") "
May 28 17:13:23.647385 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.647359    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:13:23.647511 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.647449    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:13:23.647511 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.647481    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6b50cac-d931-4455-99aa-dce466196b5c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:13:23.649223 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.649194    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:13:23.649294 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.649269    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:13:23.649504 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.649478    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-kube-api-access-6qxck" (OuterVolumeSpecName: "kube-api-access-6qxck") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "kube-api-access-6qxck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:13:23.649547 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.649503    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a6b50cac-d931-4455-99aa-dce466196b5c" (UID: "a6b50cac-d931-4455-99aa-dce466196b5c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:13:23.747699 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747676    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qxck\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-kube-api-access-6qxck\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:23.747699 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747696    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-registry-certificates\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:23.747820 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747706    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-image-registry-private-configuration\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:23.747820 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747715    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6b50cac-d931-4455-99aa-dce466196b5c-ca-trust-extracted\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:23.747820 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747724    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-bound-sa-token\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:23.747820 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747733    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6b50cac-d931-4455-99aa-dce466196b5c-trusted-ca\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:23.747820 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:23.747741    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6b50cac-d931-4455-99aa-dce466196b5c-installation-pull-secrets\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:24.587082 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:24.587046    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-588bd6dc7f-6hpns"
May 28 17:13:24.618903 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:24.618874    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-588bd6dc7f-6hpns"]
May 28 17:13:24.626701 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:24.626679    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-588bd6dc7f-6hpns"]
May 28 17:13:24.654191 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:24.654174    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6b50cac-d931-4455-99aa-dce466196b5c-registry-tls\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:13:25.970484 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:25.970451    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6b50cac-d931-4455-99aa-dce466196b5c" path="/var/lib/kubelet/pods/a6b50cac-d931-4455-99aa-dce466196b5c/volumes"
May 28 17:13:27.588930 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:27.588894    2574 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" podUID="138b31fa-fed6-4000-b655-54d63d677b41" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 17:13:27.589272 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:27.588957    2574 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9"
May 28 17:13:27.589421 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:27.589391    2574 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"855fd36470476ca7e51f5a79ad81eceb093de62cd75bcab80c2dad195f51b4c1"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 17:13:27.589463 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:27.589441    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" podUID="138b31fa-fed6-4000-b655-54d63d677b41" containerName="service-proxy" containerID="cri-o://855fd36470476ca7e51f5a79ad81eceb093de62cd75bcab80c2dad195f51b4c1" gracePeriod=30
May 28 17:13:28.597684 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:28.597655    2574 generic.go:358] "Generic (PLEG): container finished" podID="138b31fa-fed6-4000-b655-54d63d677b41" containerID="855fd36470476ca7e51f5a79ad81eceb093de62cd75bcab80c2dad195f51b4c1" exitCode=2
May 28 17:13:28.597684 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:28.597693    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" event={"ID":"138b31fa-fed6-4000-b655-54d63d677b41","Type":"ContainerDied","Data":"855fd36470476ca7e51f5a79ad81eceb093de62cd75bcab80c2dad195f51b4c1"}
May 28 17:13:28.598214 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:28.597720    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588b847b4b-pdkh9" event={"ID":"138b31fa-fed6-4000-b655-54d63d677b41","Type":"ContainerStarted","Data":"a277f546c68731d978ea84324415f5d47bf2eff8d378236cb9ffa2630a3e647a"}
May 28 17:13:33.358780 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:33.358747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bvsbc_fd0fc226-3bdc-4e62-b9d4-0ccadcc58379/dns-node-resolver/0.log"
May 28 17:13:51.751271 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:51.751226    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:13:51.753489 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:51.753470    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a5cb798-cc5b-4fdf-9671-67201a9e7718-metrics-certs\") pod \"network-metrics-daemon-j9xtb\" (UID: \"3a5cb798-cc5b-4fdf-9671-67201a9e7718\") " pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:13:51.870226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:51.870198    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t7ndh\""
May 28 17:13:51.878160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:51.878140    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9xtb"
May 28 17:13:51.994358 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:51.994317    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j9xtb"]
May 28 17:13:51.997287 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:13:51.997262    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5cb798_cc5b_4fdf_9671_67201a9e7718.slice/crio-84eadfd51bebf6e714af6587faede3d3cf3850971562dac5cb3aafba26fd1a9c WatchSource:0}: Error finding container 84eadfd51bebf6e714af6587faede3d3cf3850971562dac5cb3aafba26fd1a9c: Status 404 returned error can't find the container with id 84eadfd51bebf6e714af6587faede3d3cf3850971562dac5cb3aafba26fd1a9c
May 28 17:13:52.653928 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:52.653892    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j9xtb" event={"ID":"3a5cb798-cc5b-4fdf-9671-67201a9e7718","Type":"ContainerStarted","Data":"84eadfd51bebf6e714af6587faede3d3cf3850971562dac5cb3aafba26fd1a9c"}
May 28 17:13:53.657348 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:53.657298    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j9xtb" event={"ID":"3a5cb798-cc5b-4fdf-9671-67201a9e7718","Type":"ContainerStarted","Data":"ad33aba66b5caf7cd08c6ebe5092daf570feea41e2de684d9a963f92eba3f701"}
May 28 17:13:53.657348 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:53.657333    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j9xtb" event={"ID":"3a5cb798-cc5b-4fdf-9671-67201a9e7718","Type":"ContainerStarted","Data":"bdfac8ca6dc5d25bfbbd58d1b5b990553e02036f07ace8eb6164547204fb7f3c"}
May 28 17:13:53.673720 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:13:53.673665    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j9xtb" podStartSLOduration=252.685885191 podStartE2EDuration="4m13.673646991s" podCreationTimestamp="2026-05-28 17:09:40 +0000 UTC" firstStartedPulling="2026-05-28 17:13:51.998937875 +0000 UTC m=+252.632850577" lastFinishedPulling="2026-05-28 17:13:52.986699668 +0000 UTC m=+253.620612377" observedRunningTime="2026-05-28 17:13:53.672692912 +0000 UTC m=+254.306605633" watchObservedRunningTime="2026-05-28 17:13:53.673646991 +0000 UTC m=+254.307559713"
May 28 17:14:23.419156 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:14:23.419112    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gzmmj" podUID="8dd0cee6-4a7d-46d3-9b45-1a2679a302ad"
May 28 17:14:23.419156 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:14:23.419148    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz" podUID="7dd33fff-69cd-4898-a731-49486277ef41"
May 28 17:14:23.729550 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:23.729478    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gzmmj"
May 28 17:14:23.729695 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:23.729478    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:14:27.300971 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.300938    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:14:27.303317 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.303291    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dd33fff-69cd-4898-a731-49486277ef41-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-8rlrz\" (UID: \"7dd33fff-69cd-4898-a731-49486277ef41\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:14:27.332811 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.332792    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-f6jtj\""
May 28 17:14:27.340740 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.340722    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"
May 28 17:14:27.402168 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.402134    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:14:27.402283 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.402219    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:14:27.404603 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.404578    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2ca10b38-5078-4ede-ae2a-5261f80affb6-cert\") pod \"ingress-canary-p7gwr\" (UID: \"2ca10b38-5078-4ede-ae2a-5261f80affb6\") " pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:14:27.404603 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.404596    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8dd0cee6-4a7d-46d3-9b45-1a2679a302ad-metrics-tls\") pod \"dns-default-gzmmj\" (UID: \"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad\") " pod="openshift-dns/dns-default-gzmmj"
May 28 17:14:27.456382 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.456352    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz"]
May 28 17:14:27.458923 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:14:27.458898    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd33fff_69cd_4898_a731_49486277ef41.slice/crio-47b83169d92cbac8e2bc9c3298773bf617fda69ac76f60591cb788b06ff88a96 WatchSource:0}: Error finding container 47b83169d92cbac8e2bc9c3298773bf617fda69ac76f60591cb788b06ff88a96: Status 404 returned error can't find the container with id 47b83169d92cbac8e2bc9c3298773bf617fda69ac76f60591cb788b06ff88a96
May 28 17:14:27.470088 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.470070    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jgwcl\""
May 28 17:14:27.478779 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.478760    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p7gwr"
May 28 17:14:27.592206 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.592120    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p7gwr"]
May 28 17:14:27.595413 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:14:27.595379    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca10b38_5078_4ede_ae2a_5261f80affb6.slice/crio-09c8b343d6e80a5bfdd7f3b336146b8d039395ef794a0b9d6e2131d26d978b4f WatchSource:0}: Error finding container 09c8b343d6e80a5bfdd7f3b336146b8d039395ef794a0b9d6e2131d26d978b4f: Status 404 returned error can't find the container with id 09c8b343d6e80a5bfdd7f3b336146b8d039395ef794a0b9d6e2131d26d978b4f
May 28 17:14:27.632094 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.632067    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmnj9\""
May 28 17:14:27.640281 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.640260    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gzmmj"
May 28 17:14:27.739795 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.739766    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7gwr" event={"ID":"2ca10b38-5078-4ede-ae2a-5261f80affb6","Type":"ContainerStarted","Data":"09c8b343d6e80a5bfdd7f3b336146b8d039395ef794a0b9d6e2131d26d978b4f"}
May 28 17:14:27.740840 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.740811    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz" event={"ID":"7dd33fff-69cd-4898-a731-49486277ef41","Type":"ContainerStarted","Data":"47b83169d92cbac8e2bc9c3298773bf617fda69ac76f60591cb788b06ff88a96"}
May 28 17:14:27.756595 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:27.756567    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gzmmj"]
May 28 17:14:27.759169 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:14:27.759144    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd0cee6_4a7d_46d3_9b45_1a2679a302ad.slice/crio-913b6e08599d93a724491bda0b886401de3f491cf6f79fd0db6dd597bbfb2340 WatchSource:0}: Error finding container 913b6e08599d93a724491bda0b886401de3f491cf6f79fd0db6dd597bbfb2340: Status 404 returned error can't find the container with id 913b6e08599d93a724491bda0b886401de3f491cf6f79fd0db6dd597bbfb2340
May 28 17:14:28.745197 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:28.745161    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz" event={"ID":"7dd33fff-69cd-4898-a731-49486277ef41","Type":"ContainerStarted","Data":"437252dad33938ca40aba5e7dbba06aea4a3a0e028f6f8f67a1ca6c7d5da497e"}
May 28 17:14:28.746476 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:28.746442    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gzmmj" event={"ID":"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad","Type":"ContainerStarted","Data":"913b6e08599d93a724491bda0b886401de3f491cf6f79fd0db6dd597bbfb2340"}
May 28 17:14:28.761373 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:28.761272    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-8rlrz" podStartSLOduration=284.581375876 podStartE2EDuration="4m45.761256863s" podCreationTimestamp="2026-05-28 17:09:43 +0000 UTC" firstStartedPulling="2026-05-28 17:14:27.460631134 +0000 UTC m=+288.094543836" lastFinishedPulling="2026-05-28 17:14:28.640512122 +0000 UTC m=+289.274424823" observedRunningTime="2026-05-28 17:14:28.760069972 +0000 UTC m=+289.393982693" watchObservedRunningTime="2026-05-28 17:14:28.761256863 +0000 UTC m=+289.395169586"
May 28 17:14:29.752075 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:29.752038    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gzmmj" event={"ID":"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad","Type":"ContainerStarted","Data":"a1a4e2031c6d4e81ca0aed841fbc551d4cf543f8814e5942c0ac7bb22d3c9f2a"}
May 28 17:14:29.753809 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:29.753782    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p7gwr" event={"ID":"2ca10b38-5078-4ede-ae2a-5261f80affb6","Type":"ContainerStarted","Data":"598dedeebfb02dbb7938ce9efee8cd5c806063385a76a522c0dbc9edb82fab57"}
May 28 17:14:29.775730 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:29.775675    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p7gwr" podStartSLOduration=250.749546198 podStartE2EDuration="4m12.775658901s" podCreationTimestamp="2026-05-28 17:10:17 +0000 UTC" firstStartedPulling="2026-05-28 17:14:27.597079556 +0000 UTC m=+288.230992256" lastFinishedPulling="2026-05-28 17:14:29.623192257 +0000 UTC m=+290.257104959" observedRunningTime="2026-05-28 17:14:29.774876652 +0000 UTC m=+290.408789374" watchObservedRunningTime="2026-05-28 17:14:29.775658901 +0000 UTC m=+290.409571627"
May 28 17:14:30.757142 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:30.757101    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gzmmj" event={"ID":"8dd0cee6-4a7d-46d3-9b45-1a2679a302ad","Type":"ContainerStarted","Data":"85977f633ba863c1e39a062fcbb079e1697f20e86f220942aeb702d49493abe8"}
May 28 17:14:30.775269 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:30.775221    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-gzmmj" podStartSLOduration=251.916189764 podStartE2EDuration="4m13.775205665s" podCreationTimestamp="2026-05-28 17:10:17 +0000 UTC" firstStartedPulling="2026-05-28 17:14:27.760825353 +0000 UTC m=+288.394738052" lastFinishedPulling="2026-05-28 17:14:29.619841242 +0000 UTC m=+290.253753953" observedRunningTime="2026-05-28 17:14:30.773426053 +0000 UTC m=+291.407338773" watchObservedRunningTime="2026-05-28 17:14:30.775205665 +0000 UTC m=+291.409118385"
May 28 17:14:31.759715 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:31.759678    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gzmmj"
May 28 17:14:39.884889 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:39.884861    2574 kubelet.go:1628] "Image garbage collection succeeded"
May 28 17:14:41.764613 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:14:41.764579    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gzmmj"
May 28 17:16:31.471729 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.471692    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-fx8cf"]
May 28 17:16:31.474615 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.474599    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.476918 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.476897    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 28 17:16:31.477583 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.477568    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 28 17:16:31.477639 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.477604    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mhr9q\""
May 28 17:16:31.481663 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.481642    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-fx8cf"]
May 28 17:16:31.628059 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.628031    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8rkv\" (UniqueName: \"kubernetes.io/projected/c4ee668e-b661-4444-8d6c-fe977f592f96-kube-api-access-w8rkv\") pod \"cert-manager-79c8d999ff-fx8cf\" (UID: \"c4ee668e-b661-4444-8d6c-fe977f592f96\") " pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.628202 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.628073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4ee668e-b661-4444-8d6c-fe977f592f96-bound-sa-token\") pod \"cert-manager-79c8d999ff-fx8cf\" (UID: \"c4ee668e-b661-4444-8d6c-fe977f592f96\") " pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.728946 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.728870    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w8rkv\" (UniqueName: \"kubernetes.io/projected/c4ee668e-b661-4444-8d6c-fe977f592f96-kube-api-access-w8rkv\") pod \"cert-manager-79c8d999ff-fx8cf\" (UID: \"c4ee668e-b661-4444-8d6c-fe977f592f96\") " pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.728946 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.728924    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4ee668e-b661-4444-8d6c-fe977f592f96-bound-sa-token\") pod \"cert-manager-79c8d999ff-fx8cf\" (UID: \"c4ee668e-b661-4444-8d6c-fe977f592f96\") " pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.738994 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.738963    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4ee668e-b661-4444-8d6c-fe977f592f96-bound-sa-token\") pod \"cert-manager-79c8d999ff-fx8cf\" (UID: \"c4ee668e-b661-4444-8d6c-fe977f592f96\") " pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.739372 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.739354    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8rkv\" (UniqueName: \"kubernetes.io/projected/c4ee668e-b661-4444-8d6c-fe977f592f96-kube-api-access-w8rkv\") pod \"cert-manager-79c8d999ff-fx8cf\" (UID: \"c4ee668e-b661-4444-8d6c-fe977f592f96\") " pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.783405 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.783386    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-fx8cf"
May 28 17:16:31.904212 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.904180    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-fx8cf"]
May 28 17:16:31.907351 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:16:31.907307    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4ee668e_b661_4444_8d6c_fe977f592f96.slice/crio-fec5a9c95d284f6d4a0bec07f336d792f8a191dc9850fef91e1464f1058f01b7 WatchSource:0}: Error finding container fec5a9c95d284f6d4a0bec07f336d792f8a191dc9850fef91e1464f1058f01b7: Status 404 returned error can't find the container with id fec5a9c95d284f6d4a0bec07f336d792f8a191dc9850fef91e1464f1058f01b7
May 28 17:16:31.909240 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:31.909223    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:16:32.046300 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:32.046272    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-fx8cf" event={"ID":"c4ee668e-b661-4444-8d6c-fe977f592f96","Type":"ContainerStarted","Data":"fec5a9c95d284f6d4a0bec07f336d792f8a191dc9850fef91e1464f1058f01b7"}
May 28 17:16:35.061628 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:35.061593    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-fx8cf" event={"ID":"c4ee668e-b661-4444-8d6c-fe977f592f96","Type":"ContainerStarted","Data":"58aaf623b272348c6ca2faee9fdcda6f986b22df323f3ec2f284a84067534663"}
May 28 17:16:35.079553 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:35.079508    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-fx8cf" podStartSLOduration=1.783778688 podStartE2EDuration="4.079496256s" podCreationTimestamp="2026-05-28 17:16:31 +0000 UTC" firstStartedPulling="2026-05-28 17:16:31.909373687 +0000 UTC m=+412.543286389" lastFinishedPulling="2026-05-28 17:16:34.205091259 +0000 UTC m=+414.839003957" observedRunningTime="2026-05-28 17:16:35.078268806 +0000 UTC m=+415.712181527" watchObservedRunningTime="2026-05-28 17:16:35.079496256 +0000 UTC m=+415.713408977"
May 28 17:16:57.325566 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.325537    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"]
May 28 17:16:57.330529 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.330509    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.336115 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.336087    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 28 17:16:57.336859 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.336836    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 28 17:16:57.337036 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.337016    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s4wrb\""
May 28 17:16:57.337199 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.337181    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 28 17:16:57.343743 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.343725    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 28 17:16:57.384538 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.384517    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"]
May 28 17:16:57.500975 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.500949    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qk4t\" (UniqueName: \"kubernetes.io/projected/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-kube-api-access-8qk4t\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.501123 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.500993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-webhook-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.501123 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.501030    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.601559 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.601497    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.601559 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.601539    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qk4t\" (UniqueName: \"kubernetes.io/projected/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-kube-api-access-8qk4t\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.601754 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.601657    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-webhook-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.603964 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.603945    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.604131 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.604109    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-webhook-cert\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.618818 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.618794    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qk4t\" (UniqueName: \"kubernetes.io/projected/8be90e0c-e237-4c0c-bcea-6022cac5d1c5-kube-api-access-8qk4t\") pod \"opendatahub-operator-controller-manager-64688695c6-5g9k2\" (UID: \"8be90e0c-e237-4c0c-bcea-6022cac5d1c5\") " pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.640492 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.640474    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:16:57.843861 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:57.843838    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"]
May 28 17:16:57.846206 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:16:57.846179    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be90e0c_e237_4c0c_bcea_6022cac5d1c5.slice/crio-60ee9c825549f7945a98c0e0e87360f5953bcfa4f22078ccedc6137a122088d6 WatchSource:0}: Error finding container 60ee9c825549f7945a98c0e0e87360f5953bcfa4f22078ccedc6137a122088d6: Status 404 returned error can't find the container with id 60ee9c825549f7945a98c0e0e87360f5953bcfa4f22078ccedc6137a122088d6
May 28 17:16:58.118311 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:16:58.118281    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2" event={"ID":"8be90e0c-e237-4c0c-bcea-6022cac5d1c5","Type":"ContainerStarted","Data":"60ee9c825549f7945a98c0e0e87360f5953bcfa4f22078ccedc6137a122088d6"}
May 28 17:17:01.127032 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:01.126999    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2" event={"ID":"8be90e0c-e237-4c0c-bcea-6022cac5d1c5","Type":"ContainerStarted","Data":"fd3cac38ecf7821899b373e1b5e6cba0204abfa756b549eeece89676d3b26ded"}
May 28 17:17:01.127422 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:01.127111    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:17:01.152028 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:01.151972    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2" podStartSLOduration=1.5821433790000001 podStartE2EDuration="4.151958325s" podCreationTimestamp="2026-05-28 17:16:57 +0000 UTC" firstStartedPulling="2026-05-28 17:16:57.847762104 +0000 UTC m=+438.481674804" lastFinishedPulling="2026-05-28 17:17:00.417577051 +0000 UTC m=+441.051489750" observedRunningTime="2026-05-28 17:17:01.150514254 +0000 UTC m=+441.784426975" watchObservedRunningTime="2026-05-28 17:17:01.151958325 +0000 UTC m=+441.785871047"
May 28 17:17:12.131189 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:12.131160    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64688695c6-5g9k2"
May 28 17:17:16.365024 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.364993    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"]
May 28 17:17:16.368330 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.368311    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.370458 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.370439    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 28 17:17:16.371179 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.371162    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4gcsf\""
May 28 17:17:16.371261 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.371197    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 28 17:17:16.371261 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.371219    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 17:17:16.371498 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.371482    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 17:17:16.377675 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.377654    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"]
May 28 17:17:16.526990 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.526954    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lmdp\" (UniqueName: \"kubernetes.io/projected/5e569469-6afd-43f9-ba0a-45979dcb3b31-kube-api-access-5lmdp\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.527158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.527004    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e569469-6afd-43f9-ba0a-45979dcb3b31-tmp\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.527158 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.527097    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e569469-6afd-43f9-ba0a-45979dcb3b31-tls-certs\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.627860 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.627777    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5lmdp\" (UniqueName: \"kubernetes.io/projected/5e569469-6afd-43f9-ba0a-45979dcb3b31-kube-api-access-5lmdp\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.627860 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.627825    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e569469-6afd-43f9-ba0a-45979dcb3b31-tmp\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.627860 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.627854    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e569469-6afd-43f9-ba0a-45979dcb3b31-tls-certs\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.630110 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.630074    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e569469-6afd-43f9-ba0a-45979dcb3b31-tmp\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.630321 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.630302    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e569469-6afd-43f9-ba0a-45979dcb3b31-tls-certs\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.636437 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.636417    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lmdp\" (UniqueName: \"kubernetes.io/projected/5e569469-6afd-43f9-ba0a-45979dcb3b31-kube-api-access-5lmdp\") pod \"kube-auth-proxy-7d599cb7c9-kxjdv\" (UID: \"5e569469-6afd-43f9-ba0a-45979dcb3b31\") " pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.679370 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.679334    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"
May 28 17:17:16.801587 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:16.801552    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv"]
May 28 17:17:16.804637 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:17:16.804613    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e569469_6afd_43f9_ba0a_45979dcb3b31.slice/crio-458193164b7f2d48df16fe379f70f15c824cf85518075c91b03c4e10db366203 WatchSource:0}: Error finding container 458193164b7f2d48df16fe379f70f15c824cf85518075c91b03c4e10db366203: Status 404 returned error can't find the container with id 458193164b7f2d48df16fe379f70f15c824cf85518075c91b03c4e10db366203
May 28 17:17:17.172251 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:17.172217    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv" event={"ID":"5e569469-6afd-43f9-ba0a-45979dcb3b31","Type":"ContainerStarted","Data":"458193164b7f2d48df16fe379f70f15c824cf85518075c91b03c4e10db366203"}
May 28 17:17:19.047962 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.047924    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-zb954"]
May 28 17:17:19.051230 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.051209    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.053326 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.053306    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tvsbl\""
May 28 17:17:19.053326 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.053323    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 28 17:17:19.057893 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.057859    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-zb954"]
May 28 17:17:19.149124 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.149093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st5zk\" (UniqueName: \"kubernetes.io/projected/377e7f8f-1c25-44ad-b860-0104acadb77d-kube-api-access-st5zk\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.149259 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.149140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.249778 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.249740    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zk\" (UniqueName: \"kubernetes.io/projected/377e7f8f-1c25-44ad-b860-0104acadb77d-kube-api-access-st5zk\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.249946 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.249798    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.249946 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:19.249903    2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 17:17:19.250040 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:19.249973    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert podName:377e7f8f-1c25-44ad-b860-0104acadb77d nodeName:}" failed. No retries permitted until 2026-05-28 17:17:19.749952086 +0000 UTC m=+460.383864799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert") pod "odh-model-controller-858dbf95b8-zb954" (UID: "377e7f8f-1c25-44ad-b860-0104acadb77d") : secret "odh-model-controller-webhook-cert" not found
May 28 17:17:19.258872 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.258844    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st5zk\" (UniqueName: \"kubernetes.io/projected/377e7f8f-1c25-44ad-b860-0104acadb77d-kube-api-access-st5zk\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.753457 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:19.753421    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:19.753610 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:19.753562    2574 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 17:17:19.753651 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:19.753623    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert podName:377e7f8f-1c25-44ad-b860-0104acadb77d nodeName:}" failed. No retries permitted until 2026-05-28 17:17:20.753609469 +0000 UTC m=+461.387522168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert") pod "odh-model-controller-858dbf95b8-zb954" (UID: "377e7f8f-1c25-44ad-b860-0104acadb77d") : secret "odh-model-controller-webhook-cert" not found
May 28 17:17:20.761304 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:20.761272    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:20.763755 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:20.763734    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/377e7f8f-1c25-44ad-b860-0104acadb77d-cert\") pod \"odh-model-controller-858dbf95b8-zb954\" (UID: \"377e7f8f-1c25-44ad-b860-0104acadb77d\") " pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:20.864244 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:20.864148    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:20.989185 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:20.989157    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-zb954"]
May 28 17:17:20.991653 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:17:20.991616    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod377e7f8f_1c25_44ad_b860_0104acadb77d.slice/crio-7a84e172634bab1d4277a7119cb60482687a7b064bec0e756ce8b4f15133e8be WatchSource:0}: Error finding container 7a84e172634bab1d4277a7119cb60482687a7b064bec0e756ce8b4f15133e8be: Status 404 returned error can't find the container with id 7a84e172634bab1d4277a7119cb60482687a7b064bec0e756ce8b4f15133e8be
May 28 17:17:21.185183 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:21.185097    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv" event={"ID":"5e569469-6afd-43f9-ba0a-45979dcb3b31","Type":"ContainerStarted","Data":"3fb991eab3501d7e78b5eeac45452b3a192d50e633b054eb1ebfcaabbf345f7a"}
May 28 17:17:21.186162 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:21.186139    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" event={"ID":"377e7f8f-1c25-44ad-b860-0104acadb77d","Type":"ContainerStarted","Data":"7a84e172634bab1d4277a7119cb60482687a7b064bec0e756ce8b4f15133e8be"}
May 28 17:17:21.201750 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:21.201568    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d599cb7c9-kxjdv" podStartSLOduration=1.4122855140000001 podStartE2EDuration="5.201553512s" podCreationTimestamp="2026-05-28 17:17:16 +0000 UTC" firstStartedPulling="2026-05-28 17:17:16.806191166 +0000 UTC m=+457.440103869" lastFinishedPulling="2026-05-28 17:17:20.59545916 +0000 UTC m=+461.229371867" observedRunningTime="2026-05-28 17:17:21.200812823 +0000 UTC m=+461.834725544" watchObservedRunningTime="2026-05-28 17:17:21.201553512 +0000 UTC m=+461.835466234"
May 28 17:17:24.201931 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:24.201847    2574 generic.go:358] "Generic (PLEG): container finished" podID="377e7f8f-1c25-44ad-b860-0104acadb77d" containerID="a9f095b2f4f08cf037cb001f873161422c02ea8a36fdc3b963ded274f0d86fac" exitCode=1
May 28 17:17:24.201931 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:24.201891    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" event={"ID":"377e7f8f-1c25-44ad-b860-0104acadb77d","Type":"ContainerDied","Data":"a9f095b2f4f08cf037cb001f873161422c02ea8a36fdc3b963ded274f0d86fac"}
May 28 17:17:24.202318 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:24.202140    2574 scope.go:117] "RemoveContainer" containerID="a9f095b2f4f08cf037cb001f873161422c02ea8a36fdc3b963ded274f0d86fac"
May 28 17:17:25.205916 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.205879    2574 generic.go:358] "Generic (PLEG): container finished" podID="377e7f8f-1c25-44ad-b860-0104acadb77d" containerID="064359581a1f339bb03f643a5aab9cbe82760ef16a8457d213e79570379239b7" exitCode=1
May 28 17:17:25.206308 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.205964    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" event={"ID":"377e7f8f-1c25-44ad-b860-0104acadb77d","Type":"ContainerDied","Data":"064359581a1f339bb03f643a5aab9cbe82760ef16a8457d213e79570379239b7"}
May 28 17:17:25.206308 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.206008    2574 scope.go:117] "RemoveContainer" containerID="a9f095b2f4f08cf037cb001f873161422c02ea8a36fdc3b963ded274f0d86fac"
May 28 17:17:25.206308 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.206197    2574 scope.go:117] "RemoveContainer" containerID="064359581a1f339bb03f643a5aab9cbe82760ef16a8457d213e79570379239b7"
May 28 17:17:25.206450 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:25.206420    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-zb954_opendatahub(377e7f8f-1c25-44ad-b860-0104acadb77d)\"" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" podUID="377e7f8f-1c25-44ad-b860-0104acadb77d"
May 28 17:17:25.316500 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.316470    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-z7l7z"]
May 28 17:17:25.320611 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.320592    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:25.322722 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.322705    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 28 17:17:25.322817 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.322792    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7xvsp\""
May 28 17:17:25.329466 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.329444    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-z7l7z"]
May 28 17:17:25.401796 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.401775    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-cert\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:25.401922 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.401830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-966cw\" (UniqueName: \"kubernetes.io/projected/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-kube-api-access-966cw\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:25.502875 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.502806    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-cert\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:25.502875 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.502866    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-966cw\" (UniqueName: \"kubernetes.io/projected/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-kube-api-access-966cw\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:25.503032 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:25.502979    2574 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 28 17:17:25.503079 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:25.503061    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-cert podName:3af1a4ca-1b42-419a-aa5a-77e91ab4f67c nodeName:}" failed. No retries permitted until 2026-05-28 17:17:26.00303865 +0000 UTC m=+466.636951355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-cert") pod "kserve-controller-manager-856948b99f-z7l7z" (UID: "3af1a4ca-1b42-419a-aa5a-77e91ab4f67c") : secret "kserve-webhook-server-cert" not found
May 28 17:17:25.523531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:25.523507    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-966cw\" (UniqueName: \"kubernetes.io/projected/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-kube-api-access-966cw\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:26.006821 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:26.006790    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-cert\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:26.009177 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:26.009136    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af1a4ca-1b42-419a-aa5a-77e91ab4f67c-cert\") pod \"kserve-controller-manager-856948b99f-z7l7z\" (UID: \"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c\") " pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:26.210188 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:26.210159    2574 scope.go:117] "RemoveContainer" containerID="064359581a1f339bb03f643a5aab9cbe82760ef16a8457d213e79570379239b7"
May 28 17:17:26.210577 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:26.210322    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-zb954_opendatahub(377e7f8f-1c25-44ad-b860-0104acadb77d)\"" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" podUID="377e7f8f-1c25-44ad-b860-0104acadb77d"
May 28 17:17:26.232646 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:26.232626    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:26.353509 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:26.353480    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-z7l7z"]
May 28 17:17:26.356550 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:17:26.356526    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af1a4ca_1b42_419a_aa5a_77e91ab4f67c.slice/crio-a797d6a17f2a70bb60f1c06ed6e5a6cc016a646d78998e8f1204dbe1dd7b4196 WatchSource:0}: Error finding container a797d6a17f2a70bb60f1c06ed6e5a6cc016a646d78998e8f1204dbe1dd7b4196: Status 404 returned error can't find the container with id a797d6a17f2a70bb60f1c06ed6e5a6cc016a646d78998e8f1204dbe1dd7b4196
May 28 17:17:27.213357 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:27.213317    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z" event={"ID":"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c","Type":"ContainerStarted","Data":"a797d6a17f2a70bb60f1c06ed6e5a6cc016a646d78998e8f1204dbe1dd7b4196"}
May 28 17:17:30.864651 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:30.864567    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:30.865021 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:30.864957    2574 scope.go:117] "RemoveContainer" containerID="064359581a1f339bb03f643a5aab9cbe82760ef16a8457d213e79570379239b7"
May 28 17:17:30.865131 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:17:30.865114    2574 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-zb954_opendatahub(377e7f8f-1c25-44ad-b860-0104acadb77d)\"" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" podUID="377e7f8f-1c25-44ad-b860-0104acadb77d"
May 28 17:17:31.226570 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:31.226487    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z" event={"ID":"3af1a4ca-1b42-419a-aa5a-77e91ab4f67c","Type":"ContainerStarted","Data":"fd0b6399c6e8a608da92041b5d356ee09cbd6e2ce9a1c4213ac601858405cdb5"}
May 28 17:17:31.226570 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:31.226554    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:17:31.241755 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:31.241713    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z" podStartSLOduration=2.056196548 podStartE2EDuration="6.241699781s" podCreationTimestamp="2026-05-28 17:17:25 +0000 UTC" firstStartedPulling="2026-05-28 17:17:26.35786108 +0000 UTC m=+466.991773778" lastFinishedPulling="2026-05-28 17:17:30.543364312 +0000 UTC m=+471.177277011" observedRunningTime="2026-05-28 17:17:31.241264595 +0000 UTC m=+471.875177318" watchObservedRunningTime="2026-05-28 17:17:31.241699781 +0000 UTC m=+471.875612501"
May 28 17:17:34.463845 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.463806    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"]
May 28 17:17:34.548469 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.548440    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"]
May 28 17:17:34.548617 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.548546    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.551277 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.551253    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 28 17:17:34.551467 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.551360    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 28 17:17:34.551467 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.551448    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-vvjh6\""
May 28 17:17:34.665881 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.665849    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-882gf\" (UniqueName: \"kubernetes.io/projected/ac1d9c96-d48e-41be-9959-b6bbdb206ccd-kube-api-access-882gf\") pod \"servicemesh-operator3-55f49c5f94-sgpk5\" (UID: \"ac1d9c96-d48e-41be-9959-b6bbdb206ccd\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.666042 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.665913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/ac1d9c96-d48e-41be-9959-b6bbdb206ccd-operator-config\") pod \"servicemesh-operator3-55f49c5f94-sgpk5\" (UID: \"ac1d9c96-d48e-41be-9959-b6bbdb206ccd\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.766438 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.766357    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/ac1d9c96-d48e-41be-9959-b6bbdb206ccd-operator-config\") pod \"servicemesh-operator3-55f49c5f94-sgpk5\" (UID: \"ac1d9c96-d48e-41be-9959-b6bbdb206ccd\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.766438 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.766415    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-882gf\" (UniqueName: \"kubernetes.io/projected/ac1d9c96-d48e-41be-9959-b6bbdb206ccd-kube-api-access-882gf\") pod \"servicemesh-operator3-55f49c5f94-sgpk5\" (UID: \"ac1d9c96-d48e-41be-9959-b6bbdb206ccd\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.768988 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.768955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/ac1d9c96-d48e-41be-9959-b6bbdb206ccd-operator-config\") pod \"servicemesh-operator3-55f49c5f94-sgpk5\" (UID: \"ac1d9c96-d48e-41be-9959-b6bbdb206ccd\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.777394 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.777363    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-882gf\" (UniqueName: \"kubernetes.io/projected/ac1d9c96-d48e-41be-9959-b6bbdb206ccd-kube-api-access-882gf\") pod \"servicemesh-operator3-55f49c5f94-sgpk5\" (UID: \"ac1d9c96-d48e-41be-9959-b6bbdb206ccd\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.857513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.857481    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:34.995857 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:34.995770    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"]
May 28 17:17:34.999087 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:17:34.999061    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1d9c96_d48e_41be_9959_b6bbdb206ccd.slice/crio-9964c2ef77c12851797cb6ef8d53f969b022079e2cb405ba3b1c6553d22286f8 WatchSource:0}: Error finding container 9964c2ef77c12851797cb6ef8d53f969b022079e2cb405ba3b1c6553d22286f8: Status 404 returned error can't find the container with id 9964c2ef77c12851797cb6ef8d53f969b022079e2cb405ba3b1c6553d22286f8
May 28 17:17:35.239855 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:35.239825    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5" event={"ID":"ac1d9c96-d48e-41be-9959-b6bbdb206ccd","Type":"ContainerStarted","Data":"9964c2ef77c12851797cb6ef8d53f969b022079e2cb405ba3b1c6553d22286f8"}
May 28 17:17:38.253304 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:38.253265    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5" event={"ID":"ac1d9c96-d48e-41be-9959-b6bbdb206ccd","Type":"ContainerStarted","Data":"5a2d4b70b6e83e431a99486fee0531d93f331f6ed6e0fb1ea743ec071fc38166"}
May 28 17:17:38.253722 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:38.253429    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:38.276558 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:38.276457    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5" podStartSLOduration=1.2506653 podStartE2EDuration="4.276439631s" podCreationTimestamp="2026-05-28 17:17:34 +0000 UTC" firstStartedPulling="2026-05-28 17:17:35.001545566 +0000 UTC m=+475.635458264" lastFinishedPulling="2026-05-28 17:17:38.027319893 +0000 UTC m=+478.661232595" observedRunningTime="2026-05-28 17:17:38.274479663 +0000 UTC m=+478.908392408" watchObservedRunningTime="2026-05-28 17:17:38.276439631 +0000 UTC m=+478.910352353"
May 28 17:17:40.020982 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.020947    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"]
May 28 17:17:40.024114 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.024098    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.026378 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.026357    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 28 17:17:40.026486 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.026419    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 28 17:17:40.026486 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.026454    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 28 17:17:40.026486 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.026397    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-gmfpm\""
May 28 17:17:40.026486 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.026394    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 28 17:17:40.038808 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.038788    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"]
May 28 17:17:40.104369 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104322    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff6b603-77fe-4c0d-96b2-ef474a77295f-local-certs\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.104513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104379    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-cacerts\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.104513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104425    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.104513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104459    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-token\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.104513 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104499    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.104657 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.104657 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.104568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gspbw\" (UniqueName: \"kubernetes.io/projected/7ff6b603-77fe-4c0d-96b2-ef474a77295f-kube-api-access-gspbw\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.205376 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.205325    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gspbw\" (UniqueName: \"kubernetes.io/projected/7ff6b603-77fe-4c0d-96b2-ef474a77295f-kube-api-access-gspbw\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.205565 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.205393    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff6b603-77fe-4c0d-96b2-ef474a77295f-local-certs\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.205565 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.205412    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-cacerts\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.205565 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.205439    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.205565 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.205463    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-token\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.205773 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.205640    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.206211 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.206185    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.206364 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.206296    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.207743 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.207722    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ff6b603-77fe-4c0d-96b2-ef474a77295f-local-certs\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.207903 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.207885    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-cacerts\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.208043 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.208025    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.208475 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.208452    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.216992 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.216968    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ff6b603-77fe-4c0d-96b2-ef474a77295f-istio-token\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.217369 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.217327    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gspbw\" (UniqueName: \"kubernetes.io/projected/7ff6b603-77fe-4c0d-96b2-ef474a77295f-kube-api-access-gspbw\") pod \"istiod-openshift-gateway-798958bb55-9tzgs\" (UID: \"7ff6b603-77fe-4c0d-96b2-ef474a77295f\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.333012 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.332983    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:40.478073 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.478029    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"]
May 28 17:17:40.479173 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:17:40.479139    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff6b603_77fe_4c0d_96b2_ef474a77295f.slice/crio-d25db8d920f20e14b6ef9c9d7189ee7a4334d5aad6693ea332a180d814cfbba8 WatchSource:0}: Error finding container d25db8d920f20e14b6ef9c9d7189ee7a4334d5aad6693ea332a180d814cfbba8: Status 404 returned error can't find the container with id d25db8d920f20e14b6ef9c9d7189ee7a4334d5aad6693ea332a180d814cfbba8
May 28 17:17:40.864843 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.864796    2574 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:40.865232 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:40.865214    2574 scope.go:117] "RemoveContainer" containerID="064359581a1f339bb03f643a5aab9cbe82760ef16a8457d213e79570379239b7"
May 28 17:17:41.265719 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:41.265686    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" event={"ID":"377e7f8f-1c25-44ad-b860-0104acadb77d","Type":"ContainerStarted","Data":"92ebff9cc4c0eb5541cbcb1544adbf56cecaf3de2644ed40a14fa53b0a104ec2"}
May 28 17:17:41.266226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:41.266201    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:17:41.267382 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:41.267354    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs" event={"ID":"7ff6b603-77fe-4c0d-96b2-ef474a77295f","Type":"ContainerStarted","Data":"d25db8d920f20e14b6ef9c9d7189ee7a4334d5aad6693ea332a180d814cfbba8"}
May 28 17:17:41.285160 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:41.285116    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-zb954" podStartSLOduration=2.104564114 podStartE2EDuration="22.285104908s" podCreationTimestamp="2026-05-28 17:17:19 +0000 UTC" firstStartedPulling="2026-05-28 17:17:20.992924065 +0000 UTC m=+461.626836764" lastFinishedPulling="2026-05-28 17:17:41.173464855 +0000 UTC m=+481.807377558" observedRunningTime="2026-05-28 17:17:41.28391763 +0000 UTC m=+481.917830351" watchObservedRunningTime="2026-05-28 17:17:41.285104908 +0000 UTC m=+481.919017629"
May 28 17:17:43.922334 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:43.922298    2574 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
May 28 17:17:43.922629 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:43.922387    2574 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
May 28 17:17:44.278167 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:44.278060    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs" event={"ID":"7ff6b603-77fe-4c0d-96b2-ef474a77295f","Type":"ContainerStarted","Data":"7d97d50d475244fb6bc3cf390e8babbb593e33b1aceb72da6e8620484703aef6"}
May 28 17:17:44.278364 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:44.278269    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:44.280025 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:44.279996    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-9tzgs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
May 28 17:17:44.280169 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:44.280052    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs" podUID="7ff6b603-77fe-4c0d-96b2-ef474a77295f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 17:17:44.304126 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:44.304077    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs" podStartSLOduration=0.863068494 podStartE2EDuration="4.304066959s" podCreationTimestamp="2026-05-28 17:17:40 +0000 UTC" firstStartedPulling="2026-05-28 17:17:40.481051728 +0000 UTC m=+481.114964429" lastFinishedPulling="2026-05-28 17:17:43.922050192 +0000 UTC m=+484.555962894" observedRunningTime="2026-05-28 17:17:44.301069451 +0000 UTC m=+484.934982174" watchObservedRunningTime="2026-05-28 17:17:44.304066959 +0000 UTC m=+484.937979680"
May 28 17:17:45.282460 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:45.282433    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-9tzgs"
May 28 17:17:49.259094 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:49.259064    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-sgpk5"
May 28 17:17:52.273054 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:17:52.273022    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-zb954"
May 28 17:18:02.234324 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:02.234291    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-z7l7z"
May 28 17:18:51.335798 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.335753    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"]
May 28 17:18:51.339121 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.339101    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:18:51.341506 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.341486    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
May 28 17:18:51.341600 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.341541    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 17:18:51.341600 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.341547    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lvm8h\""
May 28 17:18:51.342074 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.342059    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 17:18:51.348710 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.348688    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"]
May 28 17:18:51.442173 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.442140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx89m\" (UniqueName: \"kubernetes.io/projected/85811dc9-0114-4785-953c-38ffc83eed59-kube-api-access-sx89m\") pod \"dns-operator-controller-manager-648d5c98bc-bhsf8\" (UID: \"85811dc9-0114-4785-953c-38ffc83eed59\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:18:51.543308 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.543273    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sx89m\" (UniqueName: \"kubernetes.io/projected/85811dc9-0114-4785-953c-38ffc83eed59-kube-api-access-sx89m\") pod \"dns-operator-controller-manager-648d5c98bc-bhsf8\" (UID: \"85811dc9-0114-4785-953c-38ffc83eed59\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:18:51.556060 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.556039    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx89m\" (UniqueName: \"kubernetes.io/projected/85811dc9-0114-4785-953c-38ffc83eed59-kube-api-access-sx89m\") pod \"dns-operator-controller-manager-648d5c98bc-bhsf8\" (UID: \"85811dc9-0114-4785-953c-38ffc83eed59\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:18:51.649924 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.649851    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:18:51.778199 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:51.778169    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"]
May 28 17:18:51.781704 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:18:51.781672    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85811dc9_0114_4785_953c_38ffc83eed59.slice/crio-54c5c9886e5f3ce7ad056b6539bd89faf6f6ae234a965ad5c0e8cde8ec36bf4a WatchSource:0}: Error finding container 54c5c9886e5f3ce7ad056b6539bd89faf6f6ae234a965ad5c0e8cde8ec36bf4a: Status 404 returned error can't find the container with id 54c5c9886e5f3ce7ad056b6539bd89faf6f6ae234a965ad5c0e8cde8ec36bf4a
May 28 17:18:52.492481 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:52.492449    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8" event={"ID":"85811dc9-0114-4785-953c-38ffc83eed59","Type":"ContainerStarted","Data":"54c5c9886e5f3ce7ad056b6539bd89faf6f6ae234a965ad5c0e8cde8ec36bf4a"}
May 28 17:18:54.326571 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.326535    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"]
May 28 17:18:54.329647 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.329631    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:18:54.332414 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.332395    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4p8gd\""
May 28 17:18:54.349267 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.349242    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"]
May 28 17:18:54.367281 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.367258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29qk2\" (UniqueName: \"kubernetes.io/projected/b15d1fd3-739b-41d9-b1e8-79c271de1405-kube-api-access-29qk2\") pod \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" (UID: \"b15d1fd3-739b-41d9-b1e8-79c271de1405\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:18:54.468277 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.468249    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-29qk2\" (UniqueName: \"kubernetes.io/projected/b15d1fd3-739b-41d9-b1e8-79c271de1405-kube-api-access-29qk2\") pod \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" (UID: \"b15d1fd3-739b-41d9-b1e8-79c271de1405\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:18:54.476206 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.476179    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29qk2\" (UniqueName: \"kubernetes.io/projected/b15d1fd3-739b-41d9-b1e8-79c271de1405-kube-api-access-29qk2\") pod \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" (UID: \"b15d1fd3-739b-41d9-b1e8-79c271de1405\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:18:54.500239 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.500206    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8" event={"ID":"85811dc9-0114-4785-953c-38ffc83eed59","Type":"ContainerStarted","Data":"976356eeead74715f4417958489cbf8b651789d07427dbaa4ab250318db43b20"}
May 28 17:18:54.500393 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.500317    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:18:54.518579 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.518530    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8" podStartSLOduration=1.432941735 podStartE2EDuration="3.518516817s" podCreationTimestamp="2026-05-28 17:18:51 +0000 UTC" firstStartedPulling="2026-05-28 17:18:51.784237868 +0000 UTC m=+552.418150566" lastFinishedPulling="2026-05-28 17:18:53.869812936 +0000 UTC m=+554.503725648" observedRunningTime="2026-05-28 17:18:54.517573613 +0000 UTC m=+555.151486337" watchObservedRunningTime="2026-05-28 17:18:54.518516817 +0000 UTC m=+555.152429588"
May 28 17:18:54.639051 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.638975    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:18:54.777150 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:54.777119    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"]
May 28 17:18:54.780327 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:18:54.780302    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15d1fd3_739b_41d9_b1e8_79c271de1405.slice/crio-17cd1212848a99ffc8de7d86902134bb574b6280d800fa2274686fa09424690b WatchSource:0}: Error finding container 17cd1212848a99ffc8de7d86902134bb574b6280d800fa2274686fa09424690b: Status 404 returned error can't find the container with id 17cd1212848a99ffc8de7d86902134bb574b6280d800fa2274686fa09424690b
May 28 17:18:55.504484 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:55.504450    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" event={"ID":"b15d1fd3-739b-41d9-b1e8-79c271de1405","Type":"ContainerStarted","Data":"17cd1212848a99ffc8de7d86902134bb574b6280d800fa2274686fa09424690b"}
May 28 17:18:57.512735 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:57.512701    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" event={"ID":"b15d1fd3-739b-41d9-b1e8-79c271de1405","Type":"ContainerStarted","Data":"365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e"}
May 28 17:18:57.513159 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:57.512778    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:18:57.530297 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:18:57.530255    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" podStartSLOduration=1.554142275 podStartE2EDuration="3.530243582s" podCreationTimestamp="2026-05-28 17:18:54 +0000 UTC" firstStartedPulling="2026-05-28 17:18:54.782361874 +0000 UTC m=+555.416274573" lastFinishedPulling="2026-05-28 17:18:56.758463181 +0000 UTC m=+557.392375880" observedRunningTime="2026-05-28 17:18:57.529542464 +0000 UTC m=+558.163455208" watchObservedRunningTime="2026-05-28 17:18:57.530243582 +0000 UTC m=+558.164156345"
May 28 17:19:05.507152 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:05.507124    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-bhsf8"
May 28 17:19:08.517618 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:08.517587    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:19:16.400065 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.400027    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"]
May 28 17:19:16.406671 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.406652    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.409033 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.409011    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9r5b8\""
May 28 17:19:16.420449 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.420421    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"]
May 28 17:19:16.527807 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.527780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.527930 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.527815    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vszh5\" (UniqueName: \"kubernetes.io/projected/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-kube-api-access-vszh5\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.628670 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.628644    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.628776 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.628681    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vszh5\" (UniqueName: \"kubernetes.io/projected/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-kube-api-access-vszh5\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.629025 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.629006    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.652135 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.652074    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vszh5\" (UniqueName: \"kubernetes.io/projected/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-kube-api-access-vszh5\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.716407 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.716385    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:16.839161 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:16.838989    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"]
May 28 17:19:16.841886 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:19:16.841857    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b113ea9_f9ec_4c6e_9ea1_b3a77cf4e88a.slice/crio-cbbedc90c445274c7adc8c09435b14504e44a0ec84a081496c602703d21b70d0 WatchSource:0}: Error finding container cbbedc90c445274c7adc8c09435b14504e44a0ec84a081496c602703d21b70d0: Status 404 returned error can't find the container with id cbbedc90c445274c7adc8c09435b14504e44a0ec84a081496c602703d21b70d0
May 28 17:19:17.138977 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.138943    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"]
May 28 17:19:17.142485 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.142455    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"]
May 28 17:19:17.169947 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.169922    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"]
May 28 17:19:17.170171 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.170148    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" containerName="manager" containerID="cri-o://365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e" gracePeriod=2
May 28 17:19:17.179301 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.179271    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"]
May 28 17:19:17.183082 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.183062    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.191356 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.189726    2574 status_manager.go:895] "Failed to get status for pod" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" err="pods \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:17.191356 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.190571    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"]
May 28 17:19:17.202269 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.202218    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"]
May 28 17:19:17.233309 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.233280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mzfh\" (UniqueName: \"kubernetes.io/projected/34c86cd8-5502-44fa-8c80-d676ff5a56bf-kube-api-access-9mzfh\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-2q77l\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.233449 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.233323    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c86cd8-5502-44fa-8c80-d676ff5a56bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-2q77l\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.334034 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.334000    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9mzfh\" (UniqueName: \"kubernetes.io/projected/34c86cd8-5502-44fa-8c80-d676ff5a56bf-kube-api-access-9mzfh\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-2q77l\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.334201 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.334049    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c86cd8-5502-44fa-8c80-d676ff5a56bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-2q77l\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.334427 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.334402    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c86cd8-5502-44fa-8c80-d676ff5a56bf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-2q77l\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.343724 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.343700    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mzfh\" (UniqueName: \"kubernetes.io/projected/34c86cd8-5502-44fa-8c80-d676ff5a56bf-kube-api-access-9mzfh\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-2q77l\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.467383 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.467335    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:19:17.469644 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.469609    2574 status_manager.go:895] "Failed to get status for pod" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" err="pods \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:17.535597 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.535067    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29qk2\" (UniqueName: \"kubernetes.io/projected/b15d1fd3-739b-41d9-b1e8-79c271de1405-kube-api-access-29qk2\") pod \"b15d1fd3-739b-41d9-b1e8-79c271de1405\" (UID: \"b15d1fd3-739b-41d9-b1e8-79c271de1405\") "
May 28 17:19:17.537570 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.537546    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15d1fd3-739b-41d9-b1e8-79c271de1405-kube-api-access-29qk2" (OuterVolumeSpecName: "kube-api-access-29qk2") pod "b15d1fd3-739b-41d9-b1e8-79c271de1405" (UID: "b15d1fd3-739b-41d9-b1e8-79c271de1405"). InnerVolumeSpecName "kube-api-access-29qk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:19:17.547706 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.547680    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:17.575606 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.575575    2574 generic.go:358] "Generic (PLEG): container finished" podID="b15d1fd3-739b-41d9-b1e8-79c271de1405" containerID="365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e" exitCode=0
May 28 17:19:17.575752 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.575630    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg"
May 28 17:19:17.575752 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.575709    2574 scope.go:117] "RemoveContainer" containerID="365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e"
May 28 17:19:17.577984 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.577949    2574 status_manager.go:895] "Failed to get status for pod" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" err="pods \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:17.590084 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.590057    2574 status_manager.go:895] "Failed to get status for pod" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-kqzpg" err="pods \"limitador-operator-controller-manager-85c4996f8c-kqzpg\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:17.611712 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.611685    2574 scope.go:117] "RemoveContainer" containerID="365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e"
May 28 17:19:17.612098 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:19:17.612073    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e\": container with ID starting with 365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e not found: ID does not exist" containerID="365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e"
May 28 17:19:17.612171 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.612109    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e"} err="failed to get container status \"365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e\": rpc error: code = NotFound desc = could not find container \"365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e\": container with ID starting with 365b76b12cfc3e9603564748929ef760a1bacfa73756d0d3fcd3ea014341d29e not found: ID does not exist"
May 28 17:19:17.636142 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.636114    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29qk2\" (UniqueName: \"kubernetes.io/projected/b15d1fd3-739b-41d9-b1e8-79c271de1405-kube-api-access-29qk2\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:19:17.834459 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.834430    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"]
May 28 17:19:17.848535 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:19:17.848502    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c86cd8_5502_44fa_8c80_d676ff5a56bf.slice/crio-7185e6b5eca9d0da476b0623cab30ccd96fc1d2edd927a729c4b046996f5b4ad WatchSource:0}: Error finding container 7185e6b5eca9d0da476b0623cab30ccd96fc1d2edd927a729c4b046996f5b4ad: Status 404 returned error can't find the container with id 7185e6b5eca9d0da476b0623cab30ccd96fc1d2edd927a729c4b046996f5b4ad
May 28 17:19:17.975385 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:17.975329    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" path="/var/lib/kubelet/pods/b15d1fd3-739b-41d9-b1e8-79c271de1405/volumes"
May 28 17:19:18.583938 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:18.583883    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l" event={"ID":"34c86cd8-5502-44fa-8c80-d676ff5a56bf","Type":"ContainerStarted","Data":"7185e6b5eca9d0da476b0623cab30ccd96fc1d2edd927a729c4b046996f5b4ad"}
May 28 17:19:21.600500 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.600464    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l" event={"ID":"34c86cd8-5502-44fa-8c80-d676ff5a56bf","Type":"ContainerStarted","Data":"dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773"}
May 28 17:19:21.600922 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.600521    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:21.601988 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.601957    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" containerName="manager" containerID="cri-o://f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48" gracePeriod=2
May 28 17:19:21.619752 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.619724    2574 status_manager.go:895] "Failed to get status for pod" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:21.620165 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.620130    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l" podStartSLOduration=1.933291991 podStartE2EDuration="4.620116934s" podCreationTimestamp="2026-05-28 17:19:17 +0000 UTC" firstStartedPulling="2026-05-28 17:19:17.851283286 +0000 UTC m=+578.485195984" lastFinishedPulling="2026-05-28 17:19:20.538108224 +0000 UTC m=+581.172020927" observedRunningTime="2026-05-28 17:19:21.617704836 +0000 UTC m=+582.251617554" watchObservedRunningTime="2026-05-28 17:19:21.620116934 +0000 UTC m=+582.254029702"
May 28 17:19:21.826145 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.826116    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:21.828257 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.828232    2574 status_manager.go:895] "Failed to get status for pod" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:21.974415 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.974311    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vszh5\" (UniqueName: \"kubernetes.io/projected/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-kube-api-access-vszh5\") pod \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") "
May 28 17:19:21.974415 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.974369    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-extensions-socket-volume\") pod \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\" (UID: \"6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a\") "
May 28 17:19:21.974625 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.974600    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" (UID: "6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:19:21.976518 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:21.976493    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-kube-api-access-vszh5" (OuterVolumeSpecName: "kube-api-access-vszh5") pod "6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" (UID: "6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a"). InnerVolumeSpecName "kube-api-access-vszh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:19:22.074962 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.074937    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vszh5\" (UniqueName: \"kubernetes.io/projected/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-kube-api-access-vszh5\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:19:22.074962 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.074960    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a-extensions-socket-volume\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:19:22.607188 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.607149    2574 generic.go:358] "Generic (PLEG): container finished" podID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" containerID="f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48" exitCode=2
May 28 17:19:22.607597 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.607214    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj"
May 28 17:19:22.607597 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.607249    2574 scope.go:117] "RemoveContainer" containerID="f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48"
May 28 17:19:22.609602 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.609573    2574 status_manager.go:895] "Failed to get status for pod" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:22.615529 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.615514    2574 scope.go:117] "RemoveContainer" containerID="f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48"
May 28 17:19:22.615762 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:19:22.615744    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48\": container with ID starting with f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48 not found: ID does not exist" containerID="f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48"
May 28 17:19:22.615823 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.615768    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48"} err="failed to get container status \"f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48\": rpc error: code = NotFound desc = could not find container \"f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48\": container with ID starting with f5016ba4b3ac8eab9177bb5f38bf262228ed2b5cc1536936d5e7c425118b7d48 not found: ID does not exist"
May 28 17:19:22.617149 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:22.617132    2574 status_manager.go:895] "Failed to get status for pod" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-rrpsj\" is forbidden: User \"system:node:ip-10-0-141-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-155.ec2.internal' and this object"
May 28 17:19:23.970978 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:23.970946    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" path="/var/lib/kubelet/pods/6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a/volumes"
May 28 17:19:32.609770 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:32.609737    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:44.658290 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:44.658249    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"]
May 28 17:19:44.660796 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:44.658539    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l" podUID="34c86cd8-5502-44fa-8c80-d676ff5a56bf" containerName="manager" containerID="cri-o://dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773" gracePeriod=10
May 28 17:19:44.900405 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:44.900382    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:45.050790 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.050760    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c86cd8-5502-44fa-8c80-d676ff5a56bf-extensions-socket-volume\") pod \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") "
May 28 17:19:45.050942 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.050854    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mzfh\" (UniqueName: \"kubernetes.io/projected/34c86cd8-5502-44fa-8c80-d676ff5a56bf-kube-api-access-9mzfh\") pod \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\" (UID: \"34c86cd8-5502-44fa-8c80-d676ff5a56bf\") "
May 28 17:19:45.051113 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.051087    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c86cd8-5502-44fa-8c80-d676ff5a56bf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34c86cd8-5502-44fa-8c80-d676ff5a56bf" (UID: "34c86cd8-5502-44fa-8c80-d676ff5a56bf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 17:19:45.052854 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.052832    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c86cd8-5502-44fa-8c80-d676ff5a56bf-kube-api-access-9mzfh" (OuterVolumeSpecName: "kube-api-access-9mzfh") pod "34c86cd8-5502-44fa-8c80-d676ff5a56bf" (UID: "34c86cd8-5502-44fa-8c80-d676ff5a56bf"). InnerVolumeSpecName "kube-api-access-9mzfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:19:45.151413 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.151369    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c86cd8-5502-44fa-8c80-d676ff5a56bf-extensions-socket-volume\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:19:45.151413 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.151409    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mzfh\" (UniqueName: \"kubernetes.io/projected/34c86cd8-5502-44fa-8c80-d676ff5a56bf-kube-api-access-9mzfh\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:19:45.688723 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.688692    2574 generic.go:358] "Generic (PLEG): container finished" podID="34c86cd8-5502-44fa-8c80-d676ff5a56bf" containerID="dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773" exitCode=0
May 28 17:19:45.689131 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.688751    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"
May 28 17:19:45.689131 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.688766    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l" event={"ID":"34c86cd8-5502-44fa-8c80-d676ff5a56bf","Type":"ContainerDied","Data":"dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773"}
May 28 17:19:45.689131 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.688792    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l" event={"ID":"34c86cd8-5502-44fa-8c80-d676ff5a56bf","Type":"ContainerDied","Data":"7185e6b5eca9d0da476b0623cab30ccd96fc1d2edd927a729c4b046996f5b4ad"}
May 28 17:19:45.689131 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.688808    2574 scope.go:117] "RemoveContainer" containerID="dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773"
May 28 17:19:45.696789 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.696770    2574 scope.go:117] "RemoveContainer" containerID="dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773"
May 28 17:19:45.697034 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:19:45.697016    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773\": container with ID starting with dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773 not found: ID does not exist" containerID="dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773"
May 28 17:19:45.697099 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.697043    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773"} err="failed to get container status \"dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773\": rpc error: code = NotFound desc = could not find container \"dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773\": container with ID starting with dd41e4ad28844c956a4c3526b6a5e32fd0967df18139a4949209858e99b69773 not found: ID does not exist"
May 28 17:19:45.718582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.718560    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"]
May 28 17:19:45.723372 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.723334    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-2q77l"]
May 28 17:19:45.971064 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:45.970976    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c86cd8-5502-44fa-8c80-d676ff5a56bf" path="/var/lib/kubelet/pods/34c86cd8-5502-44fa-8c80-d676ff5a56bf/volumes"
May 28 17:19:49.039361 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039307    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"]
May 28 17:19:49.039709 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039604    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c86cd8-5502-44fa-8c80-d676ff5a56bf" containerName="manager"
May 28 17:19:49.039709 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039617    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c86cd8-5502-44fa-8c80-d676ff5a56bf" containerName="manager"
May 28 17:19:49.039709 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039632    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" containerName="manager"
May 28 17:19:49.039709 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039640    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" containerName="manager"
May 28 17:19:49.039709 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039655    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" containerName="manager"
May 28 17:19:49.039709 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039662    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" containerName="manager"
May 28 17:19:49.039888 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039729    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="34c86cd8-5502-44fa-8c80-d676ff5a56bf" containerName="manager"
May 28 17:19:49.039888 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039737    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b15d1fd3-739b-41d9-b1e8-79c271de1405" containerName="manager"
May 28 17:19:49.039888 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.039746    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="6b113ea9-f9ec-4c6e-9ea1-b3a77cf4e88a" containerName="manager"
May 28 17:19:49.043821 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.043800    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.047031 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.047011    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-hdbwv\""
May 28 17:19:49.064577 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.064554    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"]
May 28 17:19:49.084242 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084221    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084336 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084336 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084279    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084444 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084356    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084444 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084510 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084510 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpnlp\" (UniqueName: \"kubernetes.io/projected/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-kube-api-access-qpnlp\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084570 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084514    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.084570 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.084531    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.185705 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185676    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.185873 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185715    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.185873 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185767    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.185873 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185815    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.185873 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185842    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qpnlp\" (UniqueName: \"kubernetes.io/projected/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-kube-api-access-qpnlp\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.185873 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185865    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.186144 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185884    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.186144 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185928    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.186144 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.185952    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.187309 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.186962    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.187309 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.187260    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.187309 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.187292    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.187581 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.187414    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.194693 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.188725    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.194693 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.189393    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.194693 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.192280    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.199238 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.199212    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpnlp\" (UniqueName: \"kubernetes.io/projected/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-kube-api-access-qpnlp\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.199376 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.199260    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecb580d9-3b66-48d5-91f6-aef4198bb2cb-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-wn4px\" (UID: \"ecb580d9-3b66-48d5-91f6-aef4198bb2cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.356771 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.356688    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:49.492423 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.492398    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"]
May 28 17:19:49.494743 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:19:49.494715    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb580d9_3b66_48d5_91f6_aef4198bb2cb.slice/crio-ed5b75b98ddfacd2c82a9d8a5c931767293f8df8673d300f9fcca07c2911bfb6 WatchSource:0}: Error finding container ed5b75b98ddfacd2c82a9d8a5c931767293f8df8673d300f9fcca07c2911bfb6: Status 404 returned error can't find the container with id ed5b75b98ddfacd2c82a9d8a5c931767293f8df8673d300f9fcca07c2911bfb6
May 28 17:19:49.703607 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:49.703518    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px" event={"ID":"ecb580d9-3b66-48d5-91f6-aef4198bb2cb","Type":"ContainerStarted","Data":"ed5b75b98ddfacd2c82a9d8a5c931767293f8df8673d300f9fcca07c2911bfb6"}
May 28 17:19:52.243352 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:52.243308    2574 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
May 28 17:19:52.243597 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:52.243390    2574 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
May 28 17:19:52.243597 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:52.243420    2574 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
May 28 17:19:52.715065 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:52.715034    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px" event={"ID":"ecb580d9-3b66-48d5-91f6-aef4198bb2cb","Type":"ContainerStarted","Data":"a036768814bd29384a95ca1c409a0a11646d502d4e69b215a9c81c91b58dabe4"}
May 28 17:19:52.744823 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:52.744777    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px" podStartSLOduration=0.998285877 podStartE2EDuration="3.744763005s" podCreationTimestamp="2026-05-28 17:19:49 +0000 UTC" firstStartedPulling="2026-05-28 17:19:49.496556896 +0000 UTC m=+610.130469615" lastFinishedPulling="2026-05-28 17:19:52.243034039 +0000 UTC m=+612.876946743" observedRunningTime="2026-05-28 17:19:52.743568287 +0000 UTC m=+613.377481008" watchObservedRunningTime="2026-05-28 17:19:52.744763005 +0000 UTC m=+613.378675726"
May 28 17:19:53.356831 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:53.356799    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:53.361414 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:53.361390    2574 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:53.718477 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:53.718444    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:19:53.719319 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:19:53.719300    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-wn4px"
May 28 17:20:03.391559 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.391480    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:03.394434 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.394417    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.397003 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.396978    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 28 17:20:03.397117 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.397060    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-558v4\""
May 28 17:20:03.402686 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.402665    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:03.487851 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.487823    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:03.496655 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.496631    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-config-file\") pod \"limitador-limitador-7d549b5b-c4s22\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") " pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.496783 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.496680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2q5q\" (UniqueName: \"kubernetes.io/projected/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-kube-api-access-z2q5q\") pod \"limitador-limitador-7d549b5b-c4s22\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") " pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.597544 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.597513    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-config-file\") pod \"limitador-limitador-7d549b5b-c4s22\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") " pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.597672 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.597554    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z2q5q\" (UniqueName: \"kubernetes.io/projected/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-kube-api-access-z2q5q\") pod \"limitador-limitador-7d549b5b-c4s22\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") " pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.598084 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.598066    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-config-file\") pod \"limitador-limitador-7d549b5b-c4s22\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") " pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.607767 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.607735    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2q5q\" (UniqueName: \"kubernetes.io/projected/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-kube-api-access-z2q5q\") pod \"limitador-limitador-7d549b5b-c4s22\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") " pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.705892 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.705817    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:03.803780 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.803754    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:20:03.807759 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.807737    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:03.817713 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.817689    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:20:03.834294 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.834270    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:03.836735 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:20:03.836711    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b84786c_5bf8_4e77_9c7c_31fe64e9835e.slice/crio-fa7511eba9db75df91c0813c4e09aab4758a29558aeebece2c6fe30e95aa7bd0 WatchSource:0}: Error finding container fa7511eba9db75df91c0813c4e09aab4758a29558aeebece2c6fe30e95aa7bd0: Status 404 returned error can't find the container with id fa7511eba9db75df91c0813c4e09aab4758a29558aeebece2c6fe30e95aa7bd0
May 28 17:20:03.844077 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.844057    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:20:03.900309 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.900280    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/422f3aaa-d168-4b44-a347-08a7c25a8bee-config-file\") pod \"limitador-limitador-78c99df468-662jv\" (UID: \"422f3aaa-d168-4b44-a347-08a7c25a8bee\") " pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:03.900433 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:03.900314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjmrf\" (UniqueName: \"kubernetes.io/projected/422f3aaa-d168-4b44-a347-08a7c25a8bee-kube-api-access-fjmrf\") pod \"limitador-limitador-78c99df468-662jv\" (UID: \"422f3aaa-d168-4b44-a347-08a7c25a8bee\") " pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:04.001558 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.001478    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/422f3aaa-d168-4b44-a347-08a7c25a8bee-config-file\") pod \"limitador-limitador-78c99df468-662jv\" (UID: \"422f3aaa-d168-4b44-a347-08a7c25a8bee\") " pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:04.001558 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.001521    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fjmrf\" (UniqueName: \"kubernetes.io/projected/422f3aaa-d168-4b44-a347-08a7c25a8bee-kube-api-access-fjmrf\") pod \"limitador-limitador-78c99df468-662jv\" (UID: \"422f3aaa-d168-4b44-a347-08a7c25a8bee\") " pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:04.002039 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.002019    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/422f3aaa-d168-4b44-a347-08a7c25a8bee-config-file\") pod \"limitador-limitador-78c99df468-662jv\" (UID: \"422f3aaa-d168-4b44-a347-08a7c25a8bee\") " pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:04.010174 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.010156    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjmrf\" (UniqueName: \"kubernetes.io/projected/422f3aaa-d168-4b44-a347-08a7c25a8bee-kube-api-access-fjmrf\") pod \"limitador-limitador-78c99df468-662jv\" (UID: \"422f3aaa-d168-4b44-a347-08a7c25a8bee\") " pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:04.119806 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.119784    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:04.239813 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.239783    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:20:04.242665 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:20:04.242636    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422f3aaa_d168_4b44_a347_08a7c25a8bee.slice/crio-d4e633e3bff50f803cb8998b79c7b5d39d0dd7f9bd3c2cb02e68032ac67ee1d7 WatchSource:0}: Error finding container d4e633e3bff50f803cb8998b79c7b5d39d0dd7f9bd3c2cb02e68032ac67ee1d7: Status 404 returned error can't find the container with id d4e633e3bff50f803cb8998b79c7b5d39d0dd7f9bd3c2cb02e68032ac67ee1d7
May 28 17:20:04.777592 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.777483    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-662jv" event={"ID":"422f3aaa-d168-4b44-a347-08a7c25a8bee","Type":"ContainerStarted","Data":"d4e633e3bff50f803cb8998b79c7b5d39d0dd7f9bd3c2cb02e68032ac67ee1d7"}
May 28 17:20:04.780789 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:04.780725    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22" event={"ID":"5b84786c-5bf8-4e77-9c7c-31fe64e9835e","Type":"ContainerStarted","Data":"fa7511eba9db75df91c0813c4e09aab4758a29558aeebece2c6fe30e95aa7bd0"}
May 28 17:20:07.793096 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:07.793057    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22" event={"ID":"5b84786c-5bf8-4e77-9c7c-31fe64e9835e","Type":"ContainerStarted","Data":"3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76"}
May 28 17:20:07.793606 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:07.793284    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:07.794427 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:07.794404    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-662jv" event={"ID":"422f3aaa-d168-4b44-a347-08a7c25a8bee","Type":"ContainerStarted","Data":"5944b1db52f712e9e2aa8e8fd4f59de0ba5d6a3992d9ca743e232e406d4777e3"}
May 28 17:20:07.794613 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:07.794599    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:07.810038 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:07.809966    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22" podStartSLOduration=1.845137969 podStartE2EDuration="4.8099488s" podCreationTimestamp="2026-05-28 17:20:03 +0000 UTC" firstStartedPulling="2026-05-28 17:20:03.838495157 +0000 UTC m=+624.472407861" lastFinishedPulling="2026-05-28 17:20:06.803305979 +0000 UTC m=+627.437218692" observedRunningTime="2026-05-28 17:20:07.808936744 +0000 UTC m=+628.442849466" watchObservedRunningTime="2026-05-28 17:20:07.8099488 +0000 UTC m=+628.443861522"
May 28 17:20:07.826398 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:07.826325    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-662jv" podStartSLOduration=2.260478668 podStartE2EDuration="4.826309185s" podCreationTimestamp="2026-05-28 17:20:03 +0000 UTC" firstStartedPulling="2026-05-28 17:20:04.244359909 +0000 UTC m=+624.878272622" lastFinishedPulling="2026-05-28 17:20:06.810190436 +0000 UTC m=+627.444103139" observedRunningTime="2026-05-28 17:20:07.824733563 +0000 UTC m=+628.458646286" watchObservedRunningTime="2026-05-28 17:20:07.826309185 +0000 UTC m=+628.460221907"
May 28 17:20:18.799455 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:18.799425    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:18.799905 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:18.799476    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-662jv"
May 28 17:20:18.860676 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:18.860638    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:18.860895 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:18.860870    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22" podUID="5b84786c-5bf8-4e77-9c7c-31fe64e9835e" containerName="limitador" containerID="cri-o://3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76" gracePeriod=30
May 28 17:20:19.397567 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.397544    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:19.531197 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.531116    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-config-file\") pod \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") "
May 28 17:20:19.531383 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.531222    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2q5q\" (UniqueName: \"kubernetes.io/projected/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-kube-api-access-z2q5q\") pod \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\" (UID: \"5b84786c-5bf8-4e77-9c7c-31fe64e9835e\") "
May 28 17:20:19.531531 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.531509    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-config-file" (OuterVolumeSpecName: "config-file") pod "5b84786c-5bf8-4e77-9c7c-31fe64e9835e" (UID: "5b84786c-5bf8-4e77-9c7c-31fe64e9835e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 17:20:19.533252 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.533231    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-kube-api-access-z2q5q" (OuterVolumeSpecName: "kube-api-access-z2q5q") pod "5b84786c-5bf8-4e77-9c7c-31fe64e9835e" (UID: "5b84786c-5bf8-4e77-9c7c-31fe64e9835e"). InnerVolumeSpecName "kube-api-access-z2q5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:20:19.632548 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.632517    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2q5q\" (UniqueName: \"kubernetes.io/projected/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-kube-api-access-z2q5q\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:20:19.632548 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.632543    2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/5b84786c-5bf8-4e77-9c7c-31fe64e9835e-config-file\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:20:19.833113 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.833080    2574 generic.go:358] "Generic (PLEG): container finished" podID="5b84786c-5bf8-4e77-9c7c-31fe64e9835e" containerID="3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76" exitCode=0
May 28 17:20:19.833564 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.833136    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22"
May 28 17:20:19.833564 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.833140    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22" event={"ID":"5b84786c-5bf8-4e77-9c7c-31fe64e9835e","Type":"ContainerDied","Data":"3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76"}
May 28 17:20:19.833564 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.833171    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-c4s22" event={"ID":"5b84786c-5bf8-4e77-9c7c-31fe64e9835e","Type":"ContainerDied","Data":"fa7511eba9db75df91c0813c4e09aab4758a29558aeebece2c6fe30e95aa7bd0"}
May 28 17:20:19.833564 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.833190    2574 scope.go:117] "RemoveContainer" containerID="3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76"
May 28 17:20:19.841459 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.841436    2574 scope.go:117] "RemoveContainer" containerID="3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76"
May 28 17:20:19.841701 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:20:19.841682    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76\": container with ID starting with 3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76 not found: ID does not exist" containerID="3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76"
May 28 17:20:19.841761 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.841709    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76"} err="failed to get container status \"3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76\": rpc error: code = NotFound desc = could not find container \"3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76\": container with ID starting with 3a3ee340452feb8422b81c5ffe6e288d2cfb7fcbf6441c60c29a83e2ba3fab76 not found: ID does not exist"
May 28 17:20:19.854460 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.854437    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:19.856659 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.856639    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-c4s22"]
May 28 17:20:19.972170 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:19.972140    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b84786c-5bf8-4e77-9c7c-31fe64e9835e" path="/var/lib/kubelet/pods/5b84786c-5bf8-4e77-9c7c-31fe64e9835e/volumes"
May 28 17:20:37.372910 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.372876    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6589b49577-fw7v8"]
May 28 17:20:37.373385 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.373296    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b84786c-5bf8-4e77-9c7c-31fe64e9835e" containerName="limitador"
May 28 17:20:37.373385 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.373316    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b84786c-5bf8-4e77-9c7c-31fe64e9835e" containerName="limitador"
May 28 17:20:37.373464 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.373429    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="5b84786c-5bf8-4e77-9c7c-31fe64e9835e" containerName="limitador"
May 28 17:20:37.376475 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.376458    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:37.379049 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.379031    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bg4pn\""
May 28 17:20:37.379280 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.379254    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
May 28 17:20:37.379280 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.379263    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 17:20:37.387404 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.387377    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6589b49577-fw7v8"]
May 28 17:20:37.459483 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.459451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:37.459483 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.459486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t26l7\" (UniqueName: \"kubernetes.io/projected/bca36f61-f13e-4a35-ba03-5d9f53403857-kube-api-access-t26l7\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:37.548935 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.548899    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9c4f789f5-hjd6x"]
May 28 17:20:37.552164 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.552143    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.560153 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.560132    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:37.560271 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.560162    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t26l7\" (UniqueName: \"kubernetes.io/projected/bca36f61-f13e-4a35-ba03-5d9f53403857-kube-api-access-t26l7\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:37.560334 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:20:37.560303    2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
May 28 17:20:37.560409 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:20:37.560402    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert podName:bca36f61-f13e-4a35-ba03-5d9f53403857 nodeName:}" failed. No retries permitted until 2026-05-28 17:20:38.060381609 +0000 UTC m=+658.694294322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert") pod "maas-controller-6589b49577-fw7v8" (UID: "bca36f61-f13e-4a35-ba03-5d9f53403857") : secret "maas-controller-webhook-cert" not found
May 28 17:20:37.564841 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.564810    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-hjd6x"]
May 28 17:20:37.573378 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.573354    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t26l7\" (UniqueName: \"kubernetes.io/projected/bca36f61-f13e-4a35-ba03-5d9f53403857-kube-api-access-t26l7\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:37.661116 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.661050    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9aaf774-1343-469e-88f1-1b2eb536cf97-webhook-cert\") pod \"maas-controller-9c4f789f5-hjd6x\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") " pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.661116 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.661111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sxnn\" (UniqueName: \"kubernetes.io/projected/b9aaf774-1343-469e-88f1-1b2eb536cf97-kube-api-access-7sxnn\") pod \"maas-controller-9c4f789f5-hjd6x\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") " pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.762304 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.762276    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9aaf774-1343-469e-88f1-1b2eb536cf97-webhook-cert\") pod \"maas-controller-9c4f789f5-hjd6x\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") " pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.762446 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.762369    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7sxnn\" (UniqueName: \"kubernetes.io/projected/b9aaf774-1343-469e-88f1-1b2eb536cf97-kube-api-access-7sxnn\") pod \"maas-controller-9c4f789f5-hjd6x\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") " pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.764701 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.764674    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9aaf774-1343-469e-88f1-1b2eb536cf97-webhook-cert\") pod \"maas-controller-9c4f789f5-hjd6x\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") " pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.771036 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.771009    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sxnn\" (UniqueName: \"kubernetes.io/projected/b9aaf774-1343-469e-88f1-1b2eb536cf97-kube-api-access-7sxnn\") pod \"maas-controller-9c4f789f5-hjd6x\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") " pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:37.861262 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:37.861230    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:38.011485 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.011462    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-hjd6x"]
May 28 17:20:38.013661 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:20:38.013634    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9aaf774_1343_469e_88f1_1b2eb536cf97.slice/crio-b4d6a12e25eaaf83e4346228c67f2cc37dda8e5800a2df2f3676b4012dc6d5c2 WatchSource:0}: Error finding container b4d6a12e25eaaf83e4346228c67f2cc37dda8e5800a2df2f3676b4012dc6d5c2: Status 404 returned error can't find the container with id b4d6a12e25eaaf83e4346228c67f2cc37dda8e5800a2df2f3676b4012dc6d5c2
May 28 17:20:38.064921 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.064895    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:38.067122 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.067104    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert\") pod \"maas-controller-6589b49577-fw7v8\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") " pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:38.288429 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.288399    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:38.414546 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.414520    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6589b49577-fw7v8"]
May 28 17:20:38.416043 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:20:38.416013    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca36f61_f13e_4a35_ba03_5d9f53403857.slice/crio-4163bf320a29607a8b26b9509fc4d683fd2c8f93e5217016d715fec0dd02ea55 WatchSource:0}: Error finding container 4163bf320a29607a8b26b9509fc4d683fd2c8f93e5217016d715fec0dd02ea55: Status 404 returned error can't find the container with id 4163bf320a29607a8b26b9509fc4d683fd2c8f93e5217016d715fec0dd02ea55
May 28 17:20:38.908598 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.908562    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-fw7v8" event={"ID":"bca36f61-f13e-4a35-ba03-5d9f53403857","Type":"ContainerStarted","Data":"4163bf320a29607a8b26b9509fc4d683fd2c8f93e5217016d715fec0dd02ea55"}
May 28 17:20:38.910515 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:38.910483    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c4f789f5-hjd6x" event={"ID":"b9aaf774-1343-469e-88f1-1b2eb536cf97","Type":"ContainerStarted","Data":"b4d6a12e25eaaf83e4346228c67f2cc37dda8e5800a2df2f3676b4012dc6d5c2"}
May 28 17:20:40.839948 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:40.839913    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 17:20:40.918308 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:40.918276    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-fw7v8" event={"ID":"bca36f61-f13e-4a35-ba03-5d9f53403857","Type":"ContainerStarted","Data":"f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9"}
May 28 17:20:40.919720 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:40.919699    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c4f789f5-hjd6x" event={"ID":"b9aaf774-1343-469e-88f1-1b2eb536cf97","Type":"ContainerStarted","Data":"1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3"}
May 28 17:20:40.919837 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:40.919815    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:40.943515 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:40.943464    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-6589b49577-fw7v8" podStartSLOduration=1.515308235 podStartE2EDuration="3.943450954s" podCreationTimestamp="2026-05-28 17:20:37 +0000 UTC" firstStartedPulling="2026-05-28 17:20:38.417334283 +0000 UTC m=+659.051246985" lastFinishedPulling="2026-05-28 17:20:40.845477005 +0000 UTC m=+661.479389704" observedRunningTime="2026-05-28 17:20:40.940916244 +0000 UTC m=+661.574828965" watchObservedRunningTime="2026-05-28 17:20:40.943450954 +0000 UTC m=+661.577363676"
May 28 17:20:40.962394 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:40.962353    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-9c4f789f5-hjd6x" podStartSLOduration=1.139918924 podStartE2EDuration="3.962326397s" podCreationTimestamp="2026-05-28 17:20:37 +0000 UTC" firstStartedPulling="2026-05-28 17:20:38.014846951 +0000 UTC m=+658.648759651" lastFinishedPulling="2026-05-28 17:20:40.837254424 +0000 UTC m=+661.471167124" observedRunningTime="2026-05-28 17:20:40.960105792 +0000 UTC m=+661.594018513" watchObservedRunningTime="2026-05-28 17:20:40.962326397 +0000 UTC m=+661.596239183"
May 28 17:20:41.924728 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:41.924700    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:51.930526 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:51.930494    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:20:51.976007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:51.975977    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6589b49577-fw7v8"]
May 28 17:20:51.976250 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:51.976225    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-6589b49577-fw7v8" podUID="bca36f61-f13e-4a35-ba03-5d9f53403857" containerName="manager" containerID="cri-o://f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9" gracePeriod=30
May 28 17:20:51.984732 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:51.984700    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:52.220419 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.220396    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:52.386268 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.386229    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t26l7\" (UniqueName: \"kubernetes.io/projected/bca36f61-f13e-4a35-ba03-5d9f53403857-kube-api-access-t26l7\") pod \"bca36f61-f13e-4a35-ba03-5d9f53403857\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") "
May 28 17:20:52.386268 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.386263    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert\") pod \"bca36f61-f13e-4a35-ba03-5d9f53403857\" (UID: \"bca36f61-f13e-4a35-ba03-5d9f53403857\") "
May 28 17:20:52.388503 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.388468    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bca36f61-f13e-4a35-ba03-5d9f53403857-kube-api-access-t26l7" (OuterVolumeSpecName: "kube-api-access-t26l7") pod "bca36f61-f13e-4a35-ba03-5d9f53403857" (UID: "bca36f61-f13e-4a35-ba03-5d9f53403857"). InnerVolumeSpecName "kube-api-access-t26l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:20:52.388503 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.388494    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bca36f61-f13e-4a35-ba03-5d9f53403857" (UID: "bca36f61-f13e-4a35-ba03-5d9f53403857"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:20:52.487403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.487312    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t26l7\" (UniqueName: \"kubernetes.io/projected/bca36f61-f13e-4a35-ba03-5d9f53403857-kube-api-access-t26l7\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:20:52.487403 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.487360    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bca36f61-f13e-4a35-ba03-5d9f53403857-webhook-cert\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:20:52.961263 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.961223    2574 generic.go:358] "Generic (PLEG): container finished" podID="bca36f61-f13e-4a35-ba03-5d9f53403857" containerID="f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9" exitCode=0
May 28 17:20:52.961723 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.961280    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6589b49577-fw7v8"
May 28 17:20:52.961723 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.961306    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-fw7v8" event={"ID":"bca36f61-f13e-4a35-ba03-5d9f53403857","Type":"ContainerDied","Data":"f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9"}
May 28 17:20:52.961723 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.961363    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6589b49577-fw7v8" event={"ID":"bca36f61-f13e-4a35-ba03-5d9f53403857","Type":"ContainerDied","Data":"4163bf320a29607a8b26b9509fc4d683fd2c8f93e5217016d715fec0dd02ea55"}
May 28 17:20:52.961723 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.961381    2574 scope.go:117] "RemoveContainer" containerID="f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9"
May 28 17:20:52.969720 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.969705    2574 scope.go:117] "RemoveContainer" containerID="f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9"
May 28 17:20:52.969958 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:20:52.969938    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9\": container with ID starting with f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9 not found: ID does not exist" containerID="f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9"
May 28 17:20:52.970021 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.969971    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9"} err="failed to get container status \"f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9\": rpc error: code = NotFound desc = could not find container \"f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9\": container with ID starting with f2052814ef5142afbfab6972557d84868cddd1036bccb070ddabc46e597be9b9 not found: ID does not exist"
May 28 17:20:52.982269 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.982247    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6589b49577-fw7v8"]
May 28 17:20:52.986271 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:52.986254    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6589b49577-fw7v8"]
May 28 17:20:53.970861 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:20:53.970832    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bca36f61-f13e-4a35-ba03-5d9f53403857" path="/var/lib/kubelet/pods/bca36f61-f13e-4a35-ba03-5d9f53403857/volumes"
May 28 17:21:12.594478 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.594441    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-f9fd5994-66xw8"]
May 28 17:21:12.594931 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.594742    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bca36f61-f13e-4a35-ba03-5d9f53403857" containerName="manager"
May 28 17:21:12.594931 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.594753    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bca36f61-f13e-4a35-ba03-5d9f53403857" containerName="manager"
May 28 17:21:12.594931 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.594811    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="bca36f61-f13e-4a35-ba03-5d9f53403857" containerName="manager"
May 28 17:21:12.597718 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.597697    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:12.600650 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.600632    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-4plz9\""
May 28 17:21:12.600939 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.600918    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 28 17:21:12.609190 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.609168    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-f9fd5994-66xw8"]
May 28 17:21:12.627487 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.627463    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-maas-api-tls\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:12.627606 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.627524    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2g2q\" (UniqueName: \"kubernetes.io/projected/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-kube-api-access-x2g2q\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:12.728092 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.728046    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x2g2q\" (UniqueName: \"kubernetes.io/projected/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-kube-api-access-x2g2q\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:12.728267 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.728145    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-maas-api-tls\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:12.728358 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:21:12.728263    2574 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
May 28 17:21:12.728358 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:21:12.728353    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-maas-api-tls podName:73c909b1-6634-4ae4-9c0c-08f7b4eb4e65 nodeName:}" failed. No retries permitted until 2026-05-28 17:21:13.228316773 +0000 UTC m=+693.862229489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-maas-api-tls") pod "maas-api-f9fd5994-66xw8" (UID: "73c909b1-6634-4ae4-9c0c-08f7b4eb4e65") : secret "maas-api-serving-cert" not found
May 28 17:21:12.740474 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:12.740443    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2g2q\" (UniqueName: \"kubernetes.io/projected/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-kube-api-access-x2g2q\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:13.126857 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:13.126822    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:21:13.232064 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:13.232027    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-maas-api-tls\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:13.234872 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:13.234849    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73c909b1-6634-4ae4-9c0c-08f7b4eb4e65-maas-api-tls\") pod \"maas-api-f9fd5994-66xw8\" (UID: \"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65\") " pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:13.508226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:13.508149    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:13.638877 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:13.638817    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-f9fd5994-66xw8"]
May 28 17:21:14.028000 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:14.027968    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-f9fd5994-66xw8" event={"ID":"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65","Type":"ContainerStarted","Data":"3d4cb5fcba02b0fd9669e66d5c7c486b5792ba436021e8d1744871a01673442b"}
May 28 17:21:16.036040 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:16.036001    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-f9fd5994-66xw8" event={"ID":"73c909b1-6634-4ae4-9c0c-08f7b4eb4e65","Type":"ContainerStarted","Data":"49675cb9d1598276ea330ec852c3d36c603aa970915eed22dc72f52c9b78908d"}
May 28 17:21:16.036446 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:16.036124    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:16.055383 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:16.055326    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-f9fd5994-66xw8" podStartSLOduration=2.305640198 podStartE2EDuration="4.055312818s" podCreationTimestamp="2026-05-28 17:21:12 +0000 UTC" firstStartedPulling="2026-05-28 17:21:13.65152978 +0000 UTC m=+694.285442486" lastFinishedPulling="2026-05-28 17:21:15.401202406 +0000 UTC m=+696.035115106" observedRunningTime="2026-05-28 17:21:16.053107719 +0000 UTC m=+696.687020440" watchObservedRunningTime="2026-05-28 17:21:16.055312818 +0000 UTC m=+696.689225542"
May 28 17:21:22.044560 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:22.044533    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-f9fd5994-66xw8"
May 28 17:21:48.436678 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.436643    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"]
May 28 17:21:48.441433 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.441414    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.444466 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.444445    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 28 17:21:48.444591 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.444449    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xgqlk\""
May 28 17:21:48.445081 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.445063    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 28 17:21:48.445081 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.445074    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
May 28 17:21:48.451611 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.451592    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"]
May 28 17:21:48.526301 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.526269    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.526480 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.526308    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.526480 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.526361    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.526480 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.526460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.526631 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.526485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxhv8\" (UniqueName: \"kubernetes.io/projected/a19dc319-eca6-4d0f-a7d5-a55e188bb058-kube-api-access-lxhv8\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.526631 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.526521    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19dc319-eca6-4d0f-a7d5-a55e188bb058-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.627715 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.627677    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.627715 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.627716    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.627998 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.627774    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.627998 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.627800    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lxhv8\" (UniqueName: \"kubernetes.io/projected/a19dc319-eca6-4d0f-a7d5-a55e188bb058-kube-api-access-lxhv8\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.627998 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.627834    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19dc319-eca6-4d0f-a7d5-a55e188bb058-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.627998 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.627917    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.628225 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.628032    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.628225 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.628122    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.628322 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.628251    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.629986 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.629955    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a19dc319-eca6-4d0f-a7d5-a55e188bb058-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.630231 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.630214    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a19dc319-eca6-4d0f-a7d5-a55e188bb058-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.640039 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.640015    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxhv8\" (UniqueName: \"kubernetes.io/projected/a19dc319-eca6-4d0f-a7d5-a55e188bb058-kube-api-access-lxhv8\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-zkdwj\" (UID: \"a19dc319-eca6-4d0f-a7d5-a55e188bb058\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.752130 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.752060    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:21:48.813557 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.813522    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:21:48.910315 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:21:48.910282    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda19dc319_eca6_4d0f_a7d5_a55e188bb058.slice/crio-e6f7321ce914bc6eab6f6e4a4675a14de28540bf7ba07874297b97ccaf330898 WatchSource:0}: Error finding container e6f7321ce914bc6eab6f6e4a4675a14de28540bf7ba07874297b97ccaf330898: Status 404 returned error can't find the container with id e6f7321ce914bc6eab6f6e4a4675a14de28540bf7ba07874297b97ccaf330898
May 28 17:21:48.912301 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.912282    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:21:48.921791 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:48.921767    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"]
May 28 17:21:49.141890 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:49.141846    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj" event={"ID":"a19dc319-eca6-4d0f-a7d5-a55e188bb058","Type":"ContainerStarted","Data":"e6f7321ce914bc6eab6f6e4a4675a14de28540bf7ba07874297b97ccaf330898"}
May 28 17:21:50.324508 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.324473    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"]
May 28 17:21:50.327807 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.327789    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.330078 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.330052    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
May 28 17:21:50.337183 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.337143    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"]
May 28 17:21:50.445584 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.445538    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.445758 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.445591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.445758 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.445680    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.445758 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.445729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85x2w\" (UniqueName: \"kubernetes.io/projected/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-kube-api-access-85x2w\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.445910 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.445757    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.445910 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.445815    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.546544 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.546509    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.546544 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.546550    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.546782 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.546582    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.546782 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.546623    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-85x2w\" (UniqueName: \"kubernetes.io/projected/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-kube-api-access-85x2w\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.546782 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.546640    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.546782 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.546690    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.547051 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.547032    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.547649 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.547607    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.547748 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.547663    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.549325 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.549304    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.549748 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.549727    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.559105 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.559082    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85x2w\" (UniqueName: \"kubernetes.io/projected/3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c-kube-api-access-85x2w\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx\" (UID: \"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.638237 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.638142    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:21:50.794769 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:50.794710    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"]
May 28 17:21:51.156234 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:51.156199    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx" event={"ID":"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c","Type":"ContainerStarted","Data":"a324994906086b16692c1cef5d264acfe1aa85a3e3588675b3421d67182f2a92"}
May 28 17:21:51.198367 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:51.197837    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:21:56.183587 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:56.183501    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj" event={"ID":"a19dc319-eca6-4d0f-a7d5-a55e188bb058","Type":"ContainerStarted","Data":"7bac3eccadb278416e7512f40a0d007157f17d9e049f63d47fbc70741f04d49b"}
May 28 17:21:56.193011 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:56.192979    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx" event={"ID":"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c","Type":"ContainerStarted","Data":"979757864bd2c0a268f8cd9eb69de4d262f1ae9a018562884ee05a28239853c7"}
May 28 17:21:57.505582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:21:57.505541    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:22:02.216334 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:02.216300    2574 generic.go:358] "Generic (PLEG): container finished" podID="a19dc319-eca6-4d0f-a7d5-a55e188bb058" containerID="7bac3eccadb278416e7512f40a0d007157f17d9e049f63d47fbc70741f04d49b" exitCode=0
May 28 17:22:02.216735 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:02.216378    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj" event={"ID":"a19dc319-eca6-4d0f-a7d5-a55e188bb058","Type":"ContainerDied","Data":"7bac3eccadb278416e7512f40a0d007157f17d9e049f63d47fbc70741f04d49b"}
May 28 17:22:04.226570 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:04.226541    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj" event={"ID":"a19dc319-eca6-4d0f-a7d5-a55e188bb058","Type":"ContainerStarted","Data":"f7f911987f39e3e9483598b2234ddbe2556c1d2f36a4ab39dde7dee36336f961"}
May 28 17:22:04.226954 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:04.226761    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:22:04.227776 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:04.227753    2574 generic.go:358] "Generic (PLEG): container finished" podID="3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c" containerID="979757864bd2c0a268f8cd9eb69de4d262f1ae9a018562884ee05a28239853c7" exitCode=0
May 28 17:22:04.227857 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:04.227784    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx" event={"ID":"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c","Type":"ContainerDied","Data":"979757864bd2c0a268f8cd9eb69de4d262f1ae9a018562884ee05a28239853c7"}
May 28 17:22:04.247980 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:04.247939    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj" podStartSLOduration=1.337767752 podStartE2EDuration="16.247927471s" podCreationTimestamp="2026-05-28 17:21:48 +0000 UTC" firstStartedPulling="2026-05-28 17:21:48.912501649 +0000 UTC m=+729.546414361" lastFinishedPulling="2026-05-28 17:22:03.822661381 +0000 UTC m=+744.456574080" observedRunningTime="2026-05-28 17:22:04.246126393 +0000 UTC m=+744.880039113" watchObservedRunningTime="2026-05-28 17:22:04.247927471 +0000 UTC m=+744.881840192"
May 28 17:22:05.232248 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:05.232212    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx" event={"ID":"3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c","Type":"ContainerStarted","Data":"54fa0aeb8ab8a30ff1225dfbb86ea4f9cd54ecb467c14976b361932af1090405"}
May 28 17:22:05.232684 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:05.232547    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:22:05.251699 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:05.251655    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx" podStartSLOduration=1.40918721 podStartE2EDuration="15.251642073s" podCreationTimestamp="2026-05-28 17:21:50 +0000 UTC" firstStartedPulling="2026-05-28 17:21:50.802472733 +0000 UTC m=+731.436385446" lastFinishedPulling="2026-05-28 17:22:04.644927611 +0000 UTC m=+745.278840309" observedRunningTime="2026-05-28 17:22:05.250324246 +0000 UTC m=+745.884236979" watchObservedRunningTime="2026-05-28 17:22:05.251642073 +0000 UTC m=+745.885554793"
May 28 17:22:15.246103 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:15.246072    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-zkdwj"
May 28 17:22:16.248526 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:16.248494    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx"
May 28 17:22:40.803882 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:40.803850    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:22:51.533226 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.533189    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"]
May 28 17:22:51.536762 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.536745    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.538989 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.538963    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
May 28 17:22:51.547101 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.547077    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"]
May 28 17:22:51.574437 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.574333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w29hw\" (UniqueName: \"kubernetes.io/projected/98f81fba-426b-4624-8658-c207063aa56d-kube-api-access-w29hw\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.574566 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.574479    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.574566 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.574549    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.574667 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.574574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98f81fba-426b-4624-8658-c207063aa56d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.574667 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.574604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.574786 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.574740    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.675558 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.675524    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w29hw\" (UniqueName: \"kubernetes.io/projected/98f81fba-426b-4624-8658-c207063aa56d-kube-api-access-w29hw\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.675731 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.675579    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.675731 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.675636    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.675731 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.675660    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98f81fba-426b-4624-8658-c207063aa56d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.675731 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.675710    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.675936 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.675801    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.676193 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.676174    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.676271 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.676251    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.676532 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.676501    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.679008 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.678508    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98f81fba-426b-4624-8658-c207063aa56d-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.679008 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.678858    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98f81fba-426b-4624-8658-c207063aa56d-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.684656 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.684630    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w29hw\" (UniqueName: \"kubernetes.io/projected/98f81fba-426b-4624-8658-c207063aa56d-kube-api-access-w29hw\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w\" (UID: \"98f81fba-426b-4624-8658-c207063aa56d\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:51.848085 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:51.848052    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:52.184004 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:52.183977    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"]
May 28 17:22:52.185961 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:22:52.185928    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f81fba_426b_4624_8658_c207063aa56d.slice/crio-75d6ac54e4faeb7e9c298854051ac664fc20701046157e09442c2817ec6c5b98 WatchSource:0}: Error finding container 75d6ac54e4faeb7e9c298854051ac664fc20701046157e09442c2817ec6c5b98: Status 404 returned error can't find the container with id 75d6ac54e4faeb7e9c298854051ac664fc20701046157e09442c2817ec6c5b98
May 28 17:22:52.401909 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:52.401872    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w" event={"ID":"98f81fba-426b-4624-8658-c207063aa56d","Type":"ContainerStarted","Data":"0574dc9a122770d1d487fc13543c097180ca977061b6a8d654b4b7883ee169c0"}
May 28 17:22:52.401909 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:52.401912    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w" event={"ID":"98f81fba-426b-4624-8658-c207063aa56d","Type":"ContainerStarted","Data":"75d6ac54e4faeb7e9c298854051ac664fc20701046157e09442c2817ec6c5b98"}
May 28 17:22:52.918569 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:52.918533    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:22:58.422060 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:58.422021    2574 generic.go:358] "Generic (PLEG): container finished" podID="98f81fba-426b-4624-8658-c207063aa56d" containerID="0574dc9a122770d1d487fc13543c097180ca977061b6a8d654b4b7883ee169c0" exitCode=0
May 28 17:22:58.422452 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:58.422081    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w" event={"ID":"98f81fba-426b-4624-8658-c207063aa56d","Type":"ContainerDied","Data":"0574dc9a122770d1d487fc13543c097180ca977061b6a8d654b4b7883ee169c0"}
May 28 17:22:58.494735 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:58.494704    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:22:59.427195 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:59.427161    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w" event={"ID":"98f81fba-426b-4624-8658-c207063aa56d","Type":"ContainerStarted","Data":"33dcdb505e3aa22505f132138bfb745009714a7613fc3df9d93cd48bb35dd8bf"}
May 28 17:22:59.427566 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:59.427384    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:22:59.451519 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:22:59.451432    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w" podStartSLOduration=7.69201366 podStartE2EDuration="8.451419303s" podCreationTimestamp="2026-05-28 17:22:51 +0000 UTC" firstStartedPulling="2026-05-28 17:22:58.422697154 +0000 UTC m=+799.056609856" lastFinishedPulling="2026-05-28 17:22:59.182102798 +0000 UTC m=+799.816015499" observedRunningTime="2026-05-28 17:22:59.449574197 +0000 UTC m=+800.083486920" watchObservedRunningTime="2026-05-28 17:22:59.451419303 +0000 UTC m=+800.085332024"
May 28 17:23:06.594156 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:06.594118    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:23:10.444494 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:10.444461    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w"
May 28 17:23:15.807557 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:15.807518    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:23:25.498254 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:25.498222    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:23:35.092089 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:35.092053    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:23:44.788708 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:44.788670    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:23:52.325599 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.325560    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-hjd6x"]
May 28 17:23:52.326116 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.325852    2574 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-9c4f789f5-hjd6x" podUID="b9aaf774-1343-469e-88f1-1b2eb536cf97" containerName="manager" containerID="cri-o://1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3" gracePeriod=30
May 28 17:23:52.579164 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.579109    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:23:52.609557 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.609519    2574 generic.go:358] "Generic (PLEG): container finished" podID="b9aaf774-1343-469e-88f1-1b2eb536cf97" containerID="1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3" exitCode=0
May 28 17:23:52.609687 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.609588    2574 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c4f789f5-hjd6x"
May 28 17:23:52.609687 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.609580    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c4f789f5-hjd6x" event={"ID":"b9aaf774-1343-469e-88f1-1b2eb536cf97","Type":"ContainerDied","Data":"1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3"}
May 28 17:23:52.609762 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.609696    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c4f789f5-hjd6x" event={"ID":"b9aaf774-1343-469e-88f1-1b2eb536cf97","Type":"ContainerDied","Data":"b4d6a12e25eaaf83e4346228c67f2cc37dda8e5800a2df2f3676b4012dc6d5c2"}
May 28 17:23:52.609762 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.609718    2574 scope.go:117] "RemoveContainer" containerID="1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3"
May 28 17:23:52.618595 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.618578    2574 scope.go:117] "RemoveContainer" containerID="1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3"
May 28 17:23:52.618857 ip-10-0-141-155 kubenswrapper[2574]: E0528 17:23:52.618836    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3\": container with ID starting with 1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3 not found: ID does not exist" containerID="1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3"
May 28 17:23:52.618964 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.618872    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3"} err="failed to get container status \"1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3\": rpc error: code = NotFound desc = could not find container \"1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3\": container with ID starting with 1daa2a04764dd108f8102c03c36a7fb2ed5594111572d3d167d09469841e02f3 not found: ID does not exist"
May 28 17:23:52.687220 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.687203    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sxnn\" (UniqueName: \"kubernetes.io/projected/b9aaf774-1343-469e-88f1-1b2eb536cf97-kube-api-access-7sxnn\") pod \"b9aaf774-1343-469e-88f1-1b2eb536cf97\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") "
May 28 17:23:52.687365 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.687259    2574 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9aaf774-1343-469e-88f1-1b2eb536cf97-webhook-cert\") pod \"b9aaf774-1343-469e-88f1-1b2eb536cf97\" (UID: \"b9aaf774-1343-469e-88f1-1b2eb536cf97\") "
May 28 17:23:52.689087 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.689050    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9aaf774-1343-469e-88f1-1b2eb536cf97-kube-api-access-7sxnn" (OuterVolumeSpecName: "kube-api-access-7sxnn") pod "b9aaf774-1343-469e-88f1-1b2eb536cf97" (UID: "b9aaf774-1343-469e-88f1-1b2eb536cf97"). InnerVolumeSpecName "kube-api-access-7sxnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 17:23:52.689171 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.689108    2574 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9aaf774-1343-469e-88f1-1b2eb536cf97-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b9aaf774-1343-469e-88f1-1b2eb536cf97" (UID: "b9aaf774-1343-469e-88f1-1b2eb536cf97"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 17:23:52.788772 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.788750    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sxnn\" (UniqueName: \"kubernetes.io/projected/b9aaf774-1343-469e-88f1-1b2eb536cf97-kube-api-access-7sxnn\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:23:52.788772 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.788773    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9aaf774-1343-469e-88f1-1b2eb536cf97-webhook-cert\") on node \"ip-10-0-141-155.ec2.internal\" DevicePath \"\""
May 28 17:23:52.931291 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.931266    2574 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-hjd6x"]
May 28 17:23:52.935660 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:52.935640    2574 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-hjd6x"]
May 28 17:23:53.688704 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.688670    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-9c4f789f5-sqkhq"]
May 28 17:23:53.689065 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.688956    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9aaf774-1343-469e-88f1-1b2eb536cf97" containerName="manager"
May 28 17:23:53.689065 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.688968    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9aaf774-1343-469e-88f1-1b2eb536cf97" containerName="manager"
May 28 17:23:53.689065 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.689026    2574 memory_manager.go:356] "RemoveStaleState removing state" podUID="b9aaf774-1343-469e-88f1-1b2eb536cf97" containerName="manager"
May 28 17:23:53.693179 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.693160    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.695747 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.695721    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bg4pn\""
May 28 17:23:53.696582 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.696565    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
May 28 17:23:53.696675 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.696619    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 17:23:53.699665 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.699626    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-sqkhq"]
May 28 17:23:53.796029 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.795997    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d71736a1-1d25-41bb-a20b-3b2e5928d6f9-webhook-cert\") pod \"maas-controller-9c4f789f5-sqkhq\" (UID: \"d71736a1-1d25-41bb-a20b-3b2e5928d6f9\") " pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.796186 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.796058    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhcsx\" (UniqueName: \"kubernetes.io/projected/d71736a1-1d25-41bb-a20b-3b2e5928d6f9-kube-api-access-dhcsx\") pod \"maas-controller-9c4f789f5-sqkhq\" (UID: \"d71736a1-1d25-41bb-a20b-3b2e5928d6f9\") " pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.897189 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.897157    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dhcsx\" (UniqueName: \"kubernetes.io/projected/d71736a1-1d25-41bb-a20b-3b2e5928d6f9-kube-api-access-dhcsx\") pod \"maas-controller-9c4f789f5-sqkhq\" (UID: \"d71736a1-1d25-41bb-a20b-3b2e5928d6f9\") " pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.897367 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.897225    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d71736a1-1d25-41bb-a20b-3b2e5928d6f9-webhook-cert\") pod \"maas-controller-9c4f789f5-sqkhq\" (UID: \"d71736a1-1d25-41bb-a20b-3b2e5928d6f9\") " pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.899587 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.899570    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d71736a1-1d25-41bb-a20b-3b2e5928d6f9-webhook-cert\") pod \"maas-controller-9c4f789f5-sqkhq\" (UID: \"d71736a1-1d25-41bb-a20b-3b2e5928d6f9\") " pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.906290 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.906260    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhcsx\" (UniqueName: \"kubernetes.io/projected/d71736a1-1d25-41bb-a20b-3b2e5928d6f9-kube-api-access-dhcsx\") pod \"maas-controller-9c4f789f5-sqkhq\" (UID: \"d71736a1-1d25-41bb-a20b-3b2e5928d6f9\") " pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:53.971573 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:53.971520    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9aaf774-1343-469e-88f1-1b2eb536cf97" path="/var/lib/kubelet/pods/b9aaf774-1343-469e-88f1-1b2eb536cf97/volumes"
May 28 17:23:54.004204 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:54.004182    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:54.125666 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:54.125643    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-9c4f789f5-sqkhq"]
May 28 17:23:54.127960 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:23:54.127931    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71736a1_1d25_41bb_a20b_3b2e5928d6f9.slice/crio-e63a55879a10770ab2c4537acc6ac83e9c665c216cea7de2ba73eba5c97ede36 WatchSource:0}: Error finding container e63a55879a10770ab2c4537acc6ac83e9c665c216cea7de2ba73eba5c97ede36: Status 404 returned error can't find the container with id e63a55879a10770ab2c4537acc6ac83e9c665c216cea7de2ba73eba5c97ede36
May 28 17:23:54.618928 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:54.618896    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c4f789f5-sqkhq" event={"ID":"d71736a1-1d25-41bb-a20b-3b2e5928d6f9","Type":"ContainerStarted","Data":"90d54ae50891b6986e8ee02a016d6f947a45ffecfe296389a9b9887fcb253e4a"}
May 28 17:23:54.618928 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:54.618931    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-9c4f789f5-sqkhq" event={"ID":"d71736a1-1d25-41bb-a20b-3b2e5928d6f9","Type":"ContainerStarted","Data":"e63a55879a10770ab2c4537acc6ac83e9c665c216cea7de2ba73eba5c97ede36"}
May 28 17:23:54.619126 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:54.619033    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:23:54.638833 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:23:54.638789    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-9c4f789f5-sqkhq" podStartSLOduration=1.269780555 podStartE2EDuration="1.63877665s" podCreationTimestamp="2026-05-28 17:23:53 +0000 UTC" firstStartedPulling="2026-05-28 17:23:54.129279652 +0000 UTC m=+854.763192350" lastFinishedPulling="2026-05-28 17:23:54.498275734 +0000 UTC m=+855.132188445" observedRunningTime="2026-05-28 17:23:54.635982238 +0000 UTC m=+855.269894956" watchObservedRunningTime="2026-05-28 17:23:54.63877665 +0000 UTC m=+855.272689371"
May 28 17:24:05.627666 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:24:05.627639    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-9c4f789f5-sqkhq"
May 28 17:24:56.992884 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:24:56.992850    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:25:13.491659 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:25:13.491621    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:25:51.916823 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:25:51.916741    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:26:08.889392 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:26:08.889288    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:26:22.998740 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:26:22.998705    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:26:39.193118 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:26:39.193085    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:27:46.899657 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:27:46.899581    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:27:55.285220 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:27:55.285184    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:28:12.298112 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:28:12.298073    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:28:20.296206 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:28:20.296172    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:28:37.989357 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:28:37.989316    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:28:45.837884 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:28:45.837850    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:29:18.586358 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:29:18.586311    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:29:27.088301 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:29:27.088260    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:29:35.888766 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:29:35.888732    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:29:44.483831 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:29:44.483802    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:29:52.699601 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:29:52.699566    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:30:00.198934 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:30:00.198892    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:30:05.087862 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:30:05.087832    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:30:08.888447 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:30:08.888410    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:30:22.013718 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:30:22.013687    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:08.499713 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:08.499672    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:14.690458 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:14.690418    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:20.688004 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:20.687970    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:29.288246 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:29.288214    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:38.496858 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:38.496825    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:46.492209 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:46.492173    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:31:55.683073 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:31:55.683038    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:04.194637 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:04.194561    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:13.897868 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:13.897833    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:21.889571 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:21.889535    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:30.992049 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:30.992015    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:39.113655 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:39.113625    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:48.789679 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:48.789641    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:32:56.990375 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:32:56.990329    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:33:05.992736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:33:05.992697    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:33:13.895159 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:33:13.895122    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:33:22.706838 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:33:22.706801    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:33:31.491193 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:33:31.491099    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:35:48.588853 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:35:48.588816    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:35:53.897319 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:35:53.897287    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:36:19.495723 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:36:19.495686    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:36:26.292310 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:36:26.292274    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:36:35.594410 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:36:35.594314    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:36:46.285997 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:36:46.285958    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:36:55.094902 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:36:55.094866    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:37:05.889461 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:37:05.889426    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:37:15.294486 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:37:15.294450    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:37:25.584459 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:37:25.584426    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:37:33.391708 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:37:33.391673    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:37:43.996323 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:37:43.996283    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:37:52.891945 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:37:52.891912    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:38:26.089527 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:38:26.089439    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:39:09.510949 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:39:09.510913    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:39:18.093395 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:39:18.093362    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:39:25.801437 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:39:25.801397    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:39:35.079543 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:39:35.079454    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:39:42.793266 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:39:42.793228    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:39:54.993932 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:39:54.993897    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:40:02.890978 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:40:02.890942    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:40:11.184793 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:40:11.184756    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:40:19.092785 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:40:19.092739    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:40:27.592693 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:40:27.592657    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:40:36.483505 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:40:36.483470    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:40:47.284894 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:40:47.284854    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:41:05.082298 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:41:05.082214    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:41:12.708535 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:41:12.708500    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:41:21.289800 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:41:21.289764    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:41:29.681537 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:41:29.681496    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:41:46.984459 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:41:46.984421    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:41:55.296189 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:41:55.296146    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:04.089899 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:04.089858    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:12.696489 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:12.696449    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:21.789961 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:21.789926    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:30.088585 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:30.088499    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:38.986356 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:38.986310    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:50.186261 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:50.186222    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:42:59.788979 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:42:59.788943    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:43:12.599113 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:43:12.599079    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:43:22.186630 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:43:22.186593    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:43:29.991487 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:43:29.991451    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:43:37.891903 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:43:37.891866    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:43:44.982935 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:43:44.982900    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:44:02.488979 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:44:02.488882    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:44:11.187964 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:44:11.187925    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:44:19.793516 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:44:19.793479    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:44:26.886359 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:44:26.886317    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:44:52.091431 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:44:52.091395    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:45:04.778860 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:04.778823    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-662jv"]
May 28 17:45:12.623539 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:12.623504    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-z7l7z_3af1a4ca-1b42-419a-aa5a-77e91ab4f67c/manager/0.log"
May 28 17:45:12.752050 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:12.752018    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-f9fd5994-66xw8_73c909b1-6634-4ae4-9c0c-08f7b4eb4e65/maas-api/0.log"
May 28 17:45:13.078179 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:13.078133    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9c4f789f5-sqkhq_d71736a1-1d25-41bb-a20b-3b2e5928d6f9/manager/0.log"
May 28 17:45:13.201252 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:13.201222    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-zb954_377e7f8f-1c25-44ad-b860-0104acadb77d/manager/2.log"
May 28 17:45:13.323694 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:13.323649    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64688695c6-5g9k2_8be90e0c-e237-4c0c-bcea-6022cac5d1c5/manager/0.log"
May 28 17:45:15.156050 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:15.155996    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-bhsf8_85811dc9-0114-4785-953c-38ffc83eed59/manager/0.log"
May 28 17:45:15.611248 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:15.611217    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-662jv_422f3aaa-d168-4b44-a347-08a7c25a8bee/limitador/0.log"
May 28 17:45:16.173599 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:16.173565    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-9tzgs_7ff6b603-77fe-4c0d-96b2-ef474a77295f/discovery/0.log"
May 28 17:45:16.283696 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:16.283666    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d599cb7c9-kxjdv_5e569469-6afd-43f9-ba0a-45979dcb3b31/kube-auth-proxy/0.log"
May 28 17:45:16.509651 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:16.509551    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-wn4px_ecb580d9-3b66-48d5-91f6-aef4198bb2cb/istio-proxy/0.log"
May 28 17:45:17.187410 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:17.187380    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w_98f81fba-426b-4624-8658-c207063aa56d/storage-initializer/0.log"
May 28 17:45:17.194306 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:17.194286    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-mv64w_98f81fba-426b-4624-8658-c207063aa56d/main/0.log"
May 28 17:45:17.301326 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:17.301297    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-zkdwj_a19dc319-eca6-4d0f-a7d5-a55e188bb058/storage-initializer/0.log"
May 28 17:45:17.309215 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:17.309189    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-zkdwj_a19dc319-eca6-4d0f-a7d5-a55e188bb058/main/0.log"
May 28 17:45:17.658007 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:17.657933    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx_3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c/storage-initializer/0.log"
May 28 17:45:17.664599 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:17.664568    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-8d4vx_3d6f8bcd-6482-41ad-b2dd-3ea386dc4f1c/main/0.log"
May 28 17:45:24.473628 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:24.473564    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tc52b_741cdce6-f910-45f0-8929-0d0bf2be6442/global-pull-secret-syncer/0.log"
May 28 17:45:24.520056 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:24.520020    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dtzs2_d08de4f8-cf60-48a9-b73a-6d41594a9b61/konnectivity-agent/0.log"
May 28 17:45:24.641962 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:24.641932    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-155.ec2.internal_7ebf03d5493f0f79f6e5e3abb3b6ee3b/haproxy/0.log"
May 28 17:45:29.075327 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:29.075247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-bhsf8_85811dc9-0114-4785-953c-38ffc83eed59/manager/0.log"
May 28 17:45:29.214051 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:29.214021    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-662jv_422f3aaa-d168-4b44-a347-08a7c25a8bee/limitador/0.log"
May 28 17:45:31.149277 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:31.149245    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gr8n4_0045c394-d007-41ed-9cee-38f5a1a9d4fd/node-exporter/0.log"
May 28 17:45:31.167235 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:31.167210    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gr8n4_0045c394-d007-41ed-9cee-38f5a1a9d4fd/kube-rbac-proxy/0.log"
May 28 17:45:31.184716 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:31.184690    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gr8n4_0045c394-d007-41ed-9cee-38f5a1a9d4fd/init-textfile/0.log"
May 28 17:45:32.741778 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.741747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-8rlrz_7dd33fff-69cd-4898-a731-49486277ef41/networking-console-plugin/0.log"
May 28 17:45:32.954281 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.954246    2574 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"]
May 28 17:45:32.959740 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.959722    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:32.962172 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.962146    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4njhz\"/\"openshift-service-ca.crt\""
May 28 17:45:32.962801 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.962778    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"]
May 28 17:45:32.962964 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.962945    2574 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4njhz\"/\"default-dockercfg-4gln2\""
May 28 17:45:32.963039 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:32.962969    2574 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4njhz\"/\"kube-root-ca.crt\""
May 28 17:45:33.028583 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.028496    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-lib-modules\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.028583 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.028535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qh6w\" (UniqueName: \"kubernetes.io/projected/af45298b-d080-4799-9650-e4dd81269bc6-kube-api-access-5qh6w\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.028583 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.028570    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-proc\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.028817 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.028666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-podres\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.028817 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.028715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-sys\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.129787 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129748    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-lib-modules\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.129787 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129788    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5qh6w\" (UniqueName: \"kubernetes.io/projected/af45298b-d080-4799-9650-e4dd81269bc6-kube-api-access-5qh6w\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130009 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129822    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-proc\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130009 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129859    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-podres\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130009 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129889    2574 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-sys\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130009 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129940    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-proc\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130009 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129966    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-sys\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130009 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.129940    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-lib-modules\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.130224 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.130010    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/af45298b-d080-4799-9650-e4dd81269bc6-podres\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.137295 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.137271    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qh6w\" (UniqueName: \"kubernetes.io/projected/af45298b-d080-4799-9650-e4dd81269bc6-kube-api-access-5qh6w\") pod \"perf-node-gather-daemonset-k8nbj\" (UID: \"af45298b-d080-4799-9650-e4dd81269bc6\") " pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.270990 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.270947    2574 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:33.400685 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.400661    2574 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"]
May 28 17:45:33.403181 ip-10-0-141-155 kubenswrapper[2574]: W0528 17:45:33.403154    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaf45298b_d080_4799_9650_e4dd81269bc6.slice/crio-e6af4dd1cff781e23d6b6b0de8e9ae7f49ba92a439700fcb9d88ee264065b8d9 WatchSource:0}: Error finding container e6af4dd1cff781e23d6b6b0de8e9ae7f49ba92a439700fcb9d88ee264065b8d9: Status 404 returned error can't find the container with id e6af4dd1cff781e23d6b6b0de8e9ae7f49ba92a439700fcb9d88ee264065b8d9
May 28 17:45:33.404705 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:33.404687    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 17:45:34.065947 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:34.065908    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj" event={"ID":"af45298b-d080-4799-9650-e4dd81269bc6","Type":"ContainerStarted","Data":"03022bdad0b867f07970f419a63c0f9839e4b8c837eedd2c2bb7ccb9bfd03282"}
May 28 17:45:34.065947 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:34.065951    2574 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj" event={"ID":"af45298b-d080-4799-9650-e4dd81269bc6","Type":"ContainerStarted","Data":"e6af4dd1cff781e23d6b6b0de8e9ae7f49ba92a439700fcb9d88ee264065b8d9"}
May 28 17:45:34.066458 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:34.066035    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:34.084394 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:34.084312    2574 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj" podStartSLOduration=2.08429779 podStartE2EDuration="2.08429779s" podCreationTimestamp="2026-05-28 17:45:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 17:45:34.083562146 +0000 UTC m=+2154.717474879" watchObservedRunningTime="2026-05-28 17:45:34.08429779 +0000 UTC m=+2154.718210510"
May 28 17:45:35.082431 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:35.082401    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gzmmj_8dd0cee6-4a7d-46d3-9b45-1a2679a302ad/dns/0.log"
May 28 17:45:35.101746 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:35.101720    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gzmmj_8dd0cee6-4a7d-46d3-9b45-1a2679a302ad/kube-rbac-proxy/0.log"
May 28 17:45:35.201769 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:35.201739    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bvsbc_fd0fc226-3bdc-4e62-b9d4-0ccadcc58379/dns-node-resolver/0.log"
May 28 17:45:35.693736 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:35.693705    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7mc88_a8cc9d5d-aa7b-42f4-aa40-11579b736fa8/node-ca/0.log"
May 28 17:45:36.603433 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:36.603399    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-9tzgs_7ff6b603-77fe-4c0d-96b2-ef474a77295f/discovery/0.log"
May 28 17:45:36.620631 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:36.620605    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d599cb7c9-kxjdv_5e569469-6afd-43f9-ba0a-45979dcb3b31/kube-auth-proxy/0.log"
May 28 17:45:36.711399 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:36.711371    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-wn4px_ecb580d9-3b66-48d5-91f6-aef4198bb2cb/istio-proxy/0.log"
May 28 17:45:37.300318 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:37.300291    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p7gwr_2ca10b38-5078-4ede-ae2a-5261f80affb6/serve-healthcheck-canary/0.log"
May 28 17:45:37.906380 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:37.906333    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-slrlm_b76ca09e-5776-4b63-91ba-687aaf80972b/kube-rbac-proxy/0.log"
May 28 17:45:37.924684 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:37.924654    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-slrlm_b76ca09e-5776-4b63-91ba-687aaf80972b/exporter/0.log"
May 28 17:45:37.943634 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:37.943607    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-slrlm_b76ca09e-5776-4b63-91ba-687aaf80972b/extractor/0.log"
May 28 17:45:39.717829 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:39.717796    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-z7l7z_3af1a4ca-1b42-419a-aa5a-77e91ab4f67c/manager/0.log"
May 28 17:45:39.747662 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:39.747631    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-f9fd5994-66xw8_73c909b1-6634-4ae4-9c0c-08f7b4eb4e65/maas-api/0.log"
May 28 17:45:39.857627 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:39.857596    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-9c4f789f5-sqkhq_d71736a1-1d25-41bb-a20b-3b2e5928d6f9/manager/0.log"
May 28 17:45:39.874491 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:39.874464    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-zb954_377e7f8f-1c25-44ad-b860-0104acadb77d/manager/1.log"
May 28 17:45:39.889181 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:39.889156    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-zb954_377e7f8f-1c25-44ad-b860-0104acadb77d/manager/2.log"
May 28 17:45:39.928548 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:39.928511    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64688695c6-5g9k2_8be90e0c-e237-4c0c-bcea-6022cac5d1c5/manager/0.log"
May 28 17:45:40.079942 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:40.079913    2574 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4njhz/perf-node-gather-daemonset-k8nbj"
May 28 17:45:46.722196 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.722157    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/kube-multus-additional-cni-plugins/0.log"
May 28 17:45:46.740838 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.740809    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/egress-router-binary-copy/0.log"
May 28 17:45:46.763296 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.763276    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/cni-plugins/0.log"
May 28 17:45:46.782289 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.782247    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/bond-cni-plugin/0.log"
May 28 17:45:46.800064 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.800048    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/routeoverride-cni/0.log"
May 28 17:45:46.817553 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.817534    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/whereabouts-cni-bincopy/0.log"
May 28 17:45:46.835335 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:46.835317    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-s9w8f_4ba637e9-8b45-407d-8e68-50133573feac/whereabouts-cni/0.log"
May 28 17:45:47.159092 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:47.159064    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d9w6x_4e72d728-8f82-4fd9-b8cf-0377519b2328/kube-multus/0.log"
May 28 17:45:47.199955 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:47.199928    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j9xtb_3a5cb798-cc5b-4fdf-9671-67201a9e7718/network-metrics-daemon/0.log"
May 28 17:45:47.217732 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:47.217707    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j9xtb_3a5cb798-cc5b-4fdf-9671-67201a9e7718/kube-rbac-proxy/0.log"
May 28 17:45:48.659638 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.659610    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/ovn-controller/0.log"
May 28 17:45:48.685181 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.685152    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/ovn-acl-logging/0.log"
May 28 17:45:48.704743 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.704721    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/kube-rbac-proxy-node/0.log"
May 28 17:45:48.723477 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.723454    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/kube-rbac-proxy-ovn-metrics/0.log"
May 28 17:45:48.740279 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.740260    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/northd/0.log"
May 28 17:45:48.760068 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.760047    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/nbdb/0.log"
May 28 17:45:48.778093 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.778073    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/sbdb/0.log"
May 28 17:45:48.864162 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:48.864139    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxzl5_98543425-257a-4fa1-b5ad-fbef8f1e40b6/ovnkube-controller/0.log"
May 28 17:45:49.966386 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:49.966334    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-sdjr4_e2083f78-5e30-4486-a9c7-cf59b8feebd1/network-check-target-container/0.log"
May 28 17:45:50.955590 ip-10-0-141-155 kubenswrapper[2574]: I0528 17:45:50.955557    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-hv7cn_3089b2b0-3c18-4a11-b853-3cdb8fcf459a/iptables-alerter/0.log"