May 28 19:43:06.575818 ip-10-0-135-191 systemd[1]: Starting Kubernetes Kubelet...
May 28 19:43:07.093437 ip-10-0-135-191 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 19:43:07.093437 ip-10-0-135-191 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 19:43:07.093437 ip-10-0-135-191 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 19:43:07.093437 ip-10-0-135-191 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 19:43:07.093437 ip-10-0-135-191 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 19:43:07.094446 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.094352    2568 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 19:43:07.097895 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097876    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 19:43:07.097895 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097895    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097899    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097902    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097906    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097909    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097911    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097914    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097919    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097923    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097926    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097929    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097933    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097936    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097939    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097942    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097944    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097947    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097950    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097953    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 19:43:07.097962 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097955    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097958    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097961    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097963    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097966    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097969    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097972    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097975    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097978    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097980    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097983    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097986    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097988    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097991    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097993    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097996    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.097998    2568 feature_gate.go:328] unrecognized feature gate: Example
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098001    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098003    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098006    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 19:43:07.098413 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098009    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098012    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098014    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098019    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098023    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098025    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098028    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098031    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098033    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098036    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098038    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098041    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098043    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098045    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098049    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098052    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098054    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098058    2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098060    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 19:43:07.098934 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098063    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098066    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098068    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098072    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098075    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098078    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098081    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098083    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098086    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098088    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098091    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098093    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098096    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098098    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098101    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098103    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098106    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098110    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098114    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098119    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 19:43:07.099509 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098122    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098124    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098127    2568 feature_gate.go:328] unrecognized feature gate: Example2
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098129    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098132    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098134    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098137    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098578    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098585    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098588    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098592    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098595    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098598    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098601    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098604    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098607    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098610    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098612    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098615    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098617    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 19:43:07.100020 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098620    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098622    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098625    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098627    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098630    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098633    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098635    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098637    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098640    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098643    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098645    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098648    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098651    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098661    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098665    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098668    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098672    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098676    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098679    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 19:43:07.100790 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098682    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098685    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098688    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098690    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098693    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098696    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098698    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098701    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098703    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098707    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098711    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098714    2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098717    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098720    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098723    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098725    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098728    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098731    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098734    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 19:43:07.101295 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098736    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098739    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098743    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098747    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098750    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098753    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098756    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098759    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098762    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098765    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098768    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098771    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098773    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098776    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098779    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098782    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098784    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098786    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098789    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098791    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 19:43:07.101778 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098794    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098797    2568 feature_gate.go:328] unrecognized feature gate: Example
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098799    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098801    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098804    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098808    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098811    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098813    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098816    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098818    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098821    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098824    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098826    2568 feature_gate.go:328] unrecognized feature gate: Example2
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098829    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.098831    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098904    2568 flags.go:64] FLAG: --address="0.0.0.0"
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098912    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098918    2568 flags.go:64] FLAG: --anonymous-auth="true"
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098922    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098931    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098934    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 19:43:07.102263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098939    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098944    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098948    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098951    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098956    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098960    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098963    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098966    2568 flags.go:64] FLAG: --cgroup-root=""
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098969    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098972    2568 flags.go:64] FLAG: --client-ca-file=""
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098975    2568 flags.go:64] FLAG: --cloud-config=""
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098978    2568 flags.go:64] FLAG: --cloud-provider="external"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098981    2568 flags.go:64] FLAG: --cluster-dns="[]"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098985    2568 flags.go:64] FLAG: --cluster-domain=""
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098988    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098991    2568 flags.go:64] FLAG: --config-dir=""
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098994    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.098998    2568 flags.go:64] FLAG: --container-log-max-files="5"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099002    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099004    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099008    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099011    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099014    2568 flags.go:64] FLAG: --contention-profiling="false"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099017    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 19:43:07.102807 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099021    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099024    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099027    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099044    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099048    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099052    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099055    2568 flags.go:64] FLAG: --enable-load-reader="false"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099059    2568 flags.go:64] FLAG: --enable-server="true"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099062    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099066    2568 flags.go:64] FLAG: --event-burst="100"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099069    2568 flags.go:64] FLAG: --event-qps="50"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099072    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099076    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099079    2568 flags.go:64] FLAG: --eviction-hard=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099083    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099086    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099089    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099093    2568 flags.go:64] FLAG: --eviction-soft=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099096    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099099    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099102    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099104    2568 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099107    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099110    2568 flags.go:64] FLAG: --fail-swap-on="true"
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099113    2568 flags.go:64] FLAG: --feature-gates=""
May 28 19:43:07.103385 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099117    2568 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099120    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099123    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099127    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099130    2568 flags.go:64] FLAG: --healthz-port="10248"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099133    2568 flags.go:64] FLAG: --help="false"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099136    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-135-191.ec2.internal"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099139    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099142    2568 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099145    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099148    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099152    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099155    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099157    2568 flags.go:64] FLAG: --image-service-endpoint=""
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099160    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099164    2568 flags.go:64] FLAG: --kube-api-burst="100"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099166    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099173    2568 flags.go:64] FLAG: --kube-api-qps="50"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099175    2568 flags.go:64] FLAG: --kube-reserved=""
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099179    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099181    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099184    2568 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099187    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099190    2568 flags.go:64] FLAG: --lock-file=""
May 28 19:43:07.104008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099193    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099196    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099199    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099204    2568 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099207    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099210    2568 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099214    2568 flags.go:64] FLAG: --logging-format="text"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099217    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099220    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099223    2568 flags.go:64] FLAG: --manifest-url=""
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099226    2568 flags.go:64] FLAG: --manifest-url-header=""
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099230    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099233    2568 flags.go:64] FLAG: --max-open-files="1000000"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099237    2568 flags.go:64] FLAG: --max-pods="110"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099241    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099244    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099247    2568 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099250    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099253    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099255    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099258    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099266    2568 flags.go:64] FLAG: --node-status-max-images="50"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099270    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099272    2568 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 19:43:07.104656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099276    2568 flags.go:64] FLAG: --pod-cidr=""
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099280    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099287    2568 flags.go:64] FLAG: --pod-manifest-path=""
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099291    2568 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099295    2568 flags.go:64] FLAG: --pods-per-core="0"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099298    2568 flags.go:64] FLAG: --port="10250"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099301    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099304    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c40c9007172de164"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099307    2568 flags.go:64] FLAG: --qos-reserved=""
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099310    2568 flags.go:64] FLAG: --read-only-port="10255"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099313    2568 flags.go:64] FLAG: --register-node="true"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099316    2568 flags.go:64] FLAG: --register-schedulable="true"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099325    2568 flags.go:64] FLAG: --register-with-taints=""
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099330    2568 flags.go:64] FLAG: --registry-burst="10"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099333    2568 flags.go:64] FLAG: --registry-qps="5"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099336    2568 flags.go:64] FLAG: --reserved-cpus=""
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099339    2568 flags.go:64] FLAG: --reserved-memory=""
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099343    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099346    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099350    2568 flags.go:64] FLAG: --rotate-certificates="false"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099352    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099355    2568 flags.go:64] FLAG: --runonce="false"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099358    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099361    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099364    2568 flags.go:64] FLAG: --seccomp-default="false"
May 28 19:43:07.105241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099367    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099370    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099373    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099376    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099379    2568 flags.go:64] FLAG: --storage-driver-password="root"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099382    2568 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099384    2568 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099387    2568 flags.go:64] FLAG: --storage-driver-user="root"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099391    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099395    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099399    2568 flags.go:64] FLAG: --system-cgroups=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099402    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099408    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099411    2568 flags.go:64] FLAG: --tls-cert-file=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099414    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099417    2568 flags.go:64] FLAG: --tls-min-version=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099420    2568 flags.go:64] FLAG: --tls-private-key-file=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099423    2568 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099426    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099429    2568 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099432    2568 flags.go:64] FLAG: --v="2"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099437    2568 flags.go:64] FLAG: --version="false"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099441    2568 flags.go:64] FLAG: --vmodule=""
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099445    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099449    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 19:43:07.105857 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099569    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099573    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099576    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099579    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099582    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099585    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099588    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099590    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099593    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099595    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099598    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099601    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099605    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099608    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099612    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099614    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099620    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099624    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099627    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 19:43:07.106443 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099630    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099632    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099635    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099638    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099641    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099645    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099648    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099651    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099654    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099656    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099659    2568 feature_gate.go:328] unrecognized feature gate: Example
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099662    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099664    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099666    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099669    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099671    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099674    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099676    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099679    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 19:43:07.106954 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099682    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099684    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099687    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099689    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099692    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099694    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099697    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099699    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099702    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099705    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099708    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099713    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099715    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099718    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099721    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099723    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099726    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099729    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099731    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099734    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 19:43:07.107444 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099736    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099739    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099741    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099744    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099746    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099749    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099751    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099754    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099756    2568 feature_gate.go:328] unrecognized feature gate: Example2
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099759    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099761    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099764    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099766    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099769    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099771    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099774    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099776    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099779    2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099781    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099784    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 19:43:07.107995 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099786    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099789    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099795    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099799    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099802    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099805    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099808    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.099810    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 19:43:07.108504 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.099819    2568 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 19:43:07.109628 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.109603    2568 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 19:43:07.109669 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.109630    2568 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 19:43:07.109699 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109683    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 19:43:07.109699 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109689    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 19:43:07.109699 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109694    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 19:43:07.109699 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109699    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109703    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109707    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109711    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109714    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109717    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109720    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109723    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109726    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109728    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109731    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109734    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109737    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109740    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109742    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109745    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109748    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109751    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109753    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109756    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 19:43:07.109801 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109759    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109762    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109765    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109767    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109769    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109772    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109774    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109777    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109780    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109782    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109785    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109787    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109790    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109792    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109797    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109800    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109802    2568 feature_gate.go:328] unrecognized feature gate: Example
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109805    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109808    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109812    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 19:43:07.110284 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109816    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109820    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109822    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109825    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109828    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109831    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109833    2568 feature_gate.go:328] unrecognized feature gate: Example2
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109837    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109840    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109843    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109845    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109848    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109851    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109853    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109856    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109859    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109861    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109864    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109866    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109868    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 19:43:07.110796 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109871    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109874    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109877    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109879    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109882    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109885    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109888    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109890    2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109893    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109896    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109898    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109901    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109903    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109906    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109908    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109911    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109914    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109916    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109919    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109922    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 19:43:07.111290 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109924    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109927    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.109929    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.109935    2568 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110045    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110050    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110053    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110056    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110059    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110062    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110065    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110068    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110070    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110073    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110075    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 19:43:07.111803 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110078    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110081    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110085    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110089    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110092    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110094    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110097    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110099    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110103    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110107    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110111    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110114    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110117    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110120    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110123    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110125    2568 feature_gate.go:328] unrecognized feature gate: Example
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110128    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110131    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110134    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 19:43:07.112197 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110136    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110139    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110142    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110144    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110147    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110149    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110152    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110154    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110157    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110159    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110162    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110165    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110167    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110170    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110172    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110175    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110178    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110181    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110184    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110186    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 19:43:07.112685 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110189    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110192    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110194    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110197    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110200    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110203    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110205    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110208    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110210    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110213    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110215    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110217    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110220    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110223    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110225    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110227    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110230    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110233    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110235    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110237    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 19:43:07.113199 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110240    2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110242    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110245    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110247    2568 feature_gate.go:328] unrecognized feature gate: Example2
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110250    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110252    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110254    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110257    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110261    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110264    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110266    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110269    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110271    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110274    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110276    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:07.110278    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 19:43:07.113694 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.110283    2568 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 19:43:07.114102 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.110430    2568 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 19:43:07.114102 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.112820    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 19:43:07.114102 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.113940    2568 server.go:1019] "Starting client certificate rotation"
May 28 19:43:07.114102 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.114034    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 19:43:07.114102 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.114071    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 19:43:07.144888 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.144850    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 19:43:07.148820 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.148793    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 19:43:07.167635 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.167461    2568 log.go:25] "Validated CRI v1 runtime API"
May 28 19:43:07.173749 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.173725    2568 log.go:25] "Validated CRI v1 image API"
May 28 19:43:07.175086 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.175068    2568 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 19:43:07.176859 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.176840    2568 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 19:43:07.180168 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.180143    2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9bb28867-1740-471c-863d-bf9320776ec4:/dev/nvme0n1p3 9ca6ba0c-8f80-40da-847c-867192d82582:/dev/nvme0n1p4]
May 28 19:43:07.180236 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.180166    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 19:43:07.184527 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.184383    2568 manager.go:217] Machine: {Timestamp:2026-05-28 19:43:07.183861033 +0000 UTC m=+0.471201929 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098808 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f5f7f7366e6be131b1dab0aaf3b10 SystemUUID:ec2f5f7f-7366-e6be-131b-1dab0aaf3b10 BootID:59c07024-9eb9-4c3b-ba64-0d30f24bc304 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:65:1d:bb:ce:37 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:65:1d:bb:ce:37 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:dd:ef:e6:c1:0c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 19:43:07.185349 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.185336    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 19:43:07.185446 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.185435    2568 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 19:43:07.187556 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.187525    2568 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 19:43:07.187714 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.187560    2568 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-191.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 19:43:07.187758 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.187725    2568 topology_manager.go:138] "Creating topology manager with none policy"
May 28 19:43:07.187758 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.187734    2568 container_manager_linux.go:306] "Creating device plugin manager"
May 28 19:43:07.187758 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.187741    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 19:43:07.188636 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.188624    2568 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 19:43:07.190863 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.190845    2568 state_mem.go:36] "Initialized new in-memory state store"
May 28 19:43:07.190997 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.190986    2568 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 19:43:07.193593 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.193575    2568 kubelet.go:491] "Attempting to sync node with API server"
May 28 19:43:07.193646 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.193601    2568 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 19:43:07.193646 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.193615    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 19:43:07.193646 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.193627    2568 kubelet.go:397] "Adding apiserver pod source"
May 28 19:43:07.193646 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.193639    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 19:43:07.194959 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.194943    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 19:43:07.195025 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.194968    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 19:43:07.200229 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.200203    2568 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 19:43:07.202819 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.202800    2568 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 19:43:07.204059 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204047    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 19:43:07.204097 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204068    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 19:43:07.204097 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204078    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 19:43:07.204097 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204087    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 19:43:07.204097 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204096    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 19:43:07.204204 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204102    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 19:43:07.204204 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204111    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 19:43:07.204204 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204116    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 19:43:07.204204 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204125    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 19:43:07.204204 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204131    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 19:43:07.204204 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204150    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 19:43:07.204828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.204817    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 19:43:07.206510 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.206452    2568 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 28 19:43:07.206510 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.206489    2568 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 28 19:43:07.207036 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.207024    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 19:43:07.207072 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.207041    2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 19:43:07.210251 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.210236    2568 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 19:43:07.210326 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.210281    2568 server.go:1295] "Started kubelet"
May 28 19:43:07.210414 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.210384    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 19:43:07.210487 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.210367    2568 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 19:43:07.210534 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.210485    2568 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 19:43:07.211069 ip-10-0-135-191 systemd[1]: Started Kubernetes Kubelet.
May 28 19:43:07.211801 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.211787    2568 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 19:43:07.216529 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.216505    2568 server.go:317] "Adding debug handlers to kubelet server"
May 28 19:43:07.218771 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.218746    2568 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-191.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 28 19:43:07.221293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.221270    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 19:43:07.222036 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222020    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 19:43:07.222495 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222458    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bw89"
May 28 19:43:07.222581 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222571    2568 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 19:43:07.222581 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222581    2568 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 19:43:07.222843 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222825    2568 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 19:43:07.222916 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222882    2568 reconstruct.go:97] "Volume reconstruction finished"
May 28 19:43:07.222916 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.222892    2568 reconciler.go:26] "Reconciler: start to sync state"
May 28 19:43:07.223011 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.218700    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-191.ec2.internal.18b3d2315dca5c9b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-191.ec2.internal,UID:ip-10-0-135-191.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-191.ec2.internal,},FirstTimestamp:2026-05-28 19:43:07.210251419 +0000 UTC m=+0.497592315,LastTimestamp:2026-05-28 19:43:07.210251419 +0000 UTC m=+0.497592315,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-191.ec2.internal,}"
May 28 19:43:07.223011 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.223005    2568 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 19:43:07.223594 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.223579    2568 factory.go:55] Registering systemd factory
May 28 19:43:07.223666 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.223608    2568 factory.go:223] Registration of the systemd container factory successfully
May 28 19:43:07.223884 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.223872    2568 factory.go:153] Registering CRI-O factory
May 28 19:43:07.223938 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.223887    2568 factory.go:223] Registration of the crio container factory successfully
May 28 19:43:07.224944 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.224923    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.225057 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.224962    2568 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 28 19:43:07.225147 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.225020    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 19:43:07.225258 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.225158    2568 factory.go:103] Registering Raw factory
May 28 19:43:07.225258 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.225171    2568 manager.go:1196] Started watching for new ooms in manager
May 28 19:43:07.225258 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.225053    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-191.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 28 19:43:07.226293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.226275    2568 manager.go:319] Starting recovery of all containers
May 28 19:43:07.228255 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.228233    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7bw89"
May 28 19:43:07.232202 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.232160    2568 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 19:43:07.236637 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.236620    2568 manager.go:324] Recovery completed
May 28 19:43:07.240863 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.240849    2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 19:43:07.243806 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.243789    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientMemory"
May 28 19:43:07.243918 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.243825    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasNoDiskPressure"
May 28 19:43:07.243918 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.243840    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientPID"
May 28 19:43:07.244396 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.244380    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 19:43:07.244487 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.244398    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 19:43:07.244487 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.244417    2568 state_mem.go:36] "Initialized new in-memory state store"
May 28 19:43:07.248001 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.247984    2568 policy_none.go:49] "None policy: Start"
May 28 19:43:07.248096 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.248014    2568 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 19:43:07.248096 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.248029    2568 state_mem.go:35] "Initializing new in-memory state store"
May 28 19:43:07.288257 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288235    2568 manager.go:341] "Starting Device Plugin manager"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.288278    2568 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288289    2568 server.go:85] "Starting device plugin registration server"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288570    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288582    2568 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288665    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288773    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.288781    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.289691    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 19:43:07.312125 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.289727    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.329019 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.328990    2568 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 19:43:07.329019 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.329026    2568 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 19:43:07.329185 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.329077    2568 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 19:43:07.329185 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.329085    2568 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 19:43:07.329185 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.329128    2568 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 19:43:07.331741 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.331715    2568 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 19:43:07.388916 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.388883    2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 19:43:07.390105 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.390082    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientMemory"
May 28 19:43:07.390264 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.390121    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasNoDiskPressure"
May 28 19:43:07.390264 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.390131    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientPID"
May 28 19:43:07.390264 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.390156    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-191.ec2.internal"
May 28 19:43:07.399873 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.399854    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-191.ec2.internal"
May 28 19:43:07.399952 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.399882    2568 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-191.ec2.internal\": node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.421057 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.421031    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.429446 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.429420    2568 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"]
May 28 19:43:07.429963 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.429832    2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 19:43:07.432709 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.432647    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientMemory"
May 28 19:43:07.432828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.432723    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasNoDiskPressure"
May 28 19:43:07.432828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.432734    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientPID"
May 28 19:43:07.433857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.433843    2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 19:43:07.434000 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.433984    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.434052 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434022    2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 19:43:07.434824 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434806    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientMemory"
May 28 19:43:07.434909 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434838    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasNoDiskPressure"
May 28 19:43:07.434909 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434852    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientMemory"
May 28 19:43:07.434909 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434869    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasNoDiskPressure"
May 28 19:43:07.434909 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434879    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientPID"
May 28 19:43:07.435043 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.434853    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientPID"
May 28 19:43:07.435960 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.435947    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.436000 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.435972    2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 19:43:07.439701 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.439685    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientMemory"
May 28 19:43:07.439798 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.439719    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasNoDiskPressure"
May 28 19:43:07.439798 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.439735    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeHasSufficientPID"
May 28 19:43:07.463341 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.463309    2568 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-191.ec2.internal\" not found" node="ip-10-0-135-191.ec2.internal"
May 28 19:43:07.467906 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.467888    2568 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-191.ec2.internal\" not found" node="ip-10-0-135-191.ec2.internal"
May 28 19:43:07.521140 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.521096    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.524405 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.524384    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b36985eef541dcd08c713a8cd1eb1e11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal\" (UID: \"b36985eef541dcd08c713a8cd1eb1e11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.524459 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.524413    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b36985eef541dcd08c713a8cd1eb1e11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal\" (UID: \"b36985eef541dcd08c713a8cd1eb1e11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.524459 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.524433    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dac381d94558e70808c6353a7e9fc8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-191.ec2.internal\" (UID: \"f0dac381d94558e70808c6353a7e9fc8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.622281 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.622214    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.625586 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.625562    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b36985eef541dcd08c713a8cd1eb1e11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal\" (UID: \"b36985eef541dcd08c713a8cd1eb1e11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.625649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.625597    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b36985eef541dcd08c713a8cd1eb1e11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal\" (UID: \"b36985eef541dcd08c713a8cd1eb1e11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.625649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.625616    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dac381d94558e70808c6353a7e9fc8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-191.ec2.internal\" (UID: \"f0dac381d94558e70808c6353a7e9fc8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.625719 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.625662    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0dac381d94558e70808c6353a7e9fc8-config\") pod \"kube-apiserver-proxy-ip-10-0-135-191.ec2.internal\" (UID: \"f0dac381d94558e70808c6353a7e9fc8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.625719 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.625673    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b36985eef541dcd08c713a8cd1eb1e11-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal\" (UID: \"b36985eef541dcd08c713a8cd1eb1e11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.625719 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.625673    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b36985eef541dcd08c713a8cd1eb1e11-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal\" (UID: \"b36985eef541dcd08c713a8cd1eb1e11\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.723049 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.723002    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.766329 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.766307    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.770179 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:07.770158    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"
May 28 19:43:07.823278 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.823228    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:07.923685 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:07.923600    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.024198 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.024161    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.113658 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.113614    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 19:43:08.114331 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.113793    2568 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 19:43:08.124874 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.124849    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.221382 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.221355    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 19:43:08.225123 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.225104    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.230448 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.230414    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 19:38:07 +0000 UTC" deadline="2027-10-21 22:37:09.446433437 +0000 UTC"
May 28 19:43:08.230448 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.230446    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12266h54m1.215990706s"
May 28 19:43:08.237220 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.237199    2568 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 19:43:08.260374 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.260335    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xm86b"
May 28 19:43:08.267423 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.267398    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xm86b"
May 28 19:43:08.316918 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:08.316871    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36985eef541dcd08c713a8cd1eb1e11.slice/crio-620c5ca2ca245bc39e6f20c035fe1b3a048643617ee16318be784f7cda01ca6a WatchSource:0}: Error finding container 620c5ca2ca245bc39e6f20c035fe1b3a048643617ee16318be784f7cda01ca6a: Status 404 returned error can't find the container with id 620c5ca2ca245bc39e6f20c035fe1b3a048643617ee16318be784f7cda01ca6a
May 28 19:43:08.317173 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:08.317153    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0dac381d94558e70808c6353a7e9fc8.slice/crio-55a48e015a08e1533607ab3570b5bf05cee332abd63aeec8b688b4a196d5c508 WatchSource:0}: Error finding container 55a48e015a08e1533607ab3570b5bf05cee332abd63aeec8b688b4a196d5c508: Status 404 returned error can't find the container with id 55a48e015a08e1533607ab3570b5bf05cee332abd63aeec8b688b4a196d5c508
May 28 19:43:08.321114 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.321100    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 19:43:08.325951 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.325933    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.332113 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.332067    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal" event={"ID":"f0dac381d94558e70808c6353a7e9fc8","Type":"ContainerStarted","Data":"55a48e015a08e1533607ab3570b5bf05cee332abd63aeec8b688b4a196d5c508"}
May 28 19:43:08.332977 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.332955    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal" event={"ID":"b36985eef541dcd08c713a8cd1eb1e11","Type":"ContainerStarted","Data":"620c5ca2ca245bc39e6f20c035fe1b3a048643617ee16318be784f7cda01ca6a"}
May 28 19:43:08.426071 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.425988    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.526563 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.526525    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.571534 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.571508    2568 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 19:43:08.627607 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:08.627571    2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-135-191.ec2.internal\" not found"
May 28 19:43:08.692505 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.692397    2568 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 19:43:08.723755 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.723726    2568 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal"
May 28 19:43:08.737932 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.737898    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 19:43:08.740134 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.740118    2568 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal"
May 28 19:43:08.748944 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.748916    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 19:43:08.761268 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:08.761231    2568 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 19:43:09.012571 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.012491    2568 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 19:43:09.195161 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.195123    2568 apiserver.go:52] "Watching apiserver"
May 28 19:43:09.201680 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.201652    2568 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 19:43:09.202872 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.202844    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal","openshift-multus/multus-additional-cni-plugins-j7t6m","openshift-multus/multus-ngpzm","openshift-network-diagnostics/network-check-target-r9l4n","openshift-network-operator/iptables-alerter-btw6b","openshift-ovn-kubernetes/ovnkube-node-rxvsp","kube-system/konnectivity-agent-c5fpz","kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gmh25","openshift-dns/node-resolver-rflwq","openshift-multus/network-metrics-daemon-wxzjd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq","openshift-image-registry/node-ca-hl88v"]
May 28 19:43:09.205378 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.205352    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.207858 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.207839    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6sqlb\""
May 28 19:43:09.207968 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.207861    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 19:43:09.208232 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.207839    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 19:43:09.210230 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.210205    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.211240 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.210636    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.213169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.212948    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:09.213169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213017    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-298hk\""
May 28 19:43:09.213169 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.213017    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:09.213169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213019    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 19:43:09.213169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213135    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-thjhm\""
May 28 19:43:09.213169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213170    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 19:43:09.213505 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213425    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 19:43:09.213505 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213489    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 19:43:09.213607 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213506    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 19:43:09.213607 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.213583    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 19:43:09.216517 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.216498    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.219054 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.218837    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:09.219054 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.218851    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d7tpp\""
May 28 19:43:09.219054 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.218905    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 19:43:09.219054 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.219029    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 19:43:09.221577 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.221558    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.221577 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.221570    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.223872 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.223857    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 19:43:09.223999 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.223894    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 19:43:09.224057 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224039    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 19:43:09.224490 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224115    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 19:43:09.224490 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224164    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 19:43:09.224490 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224209    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.224490 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224266    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:09.224490 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224380    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 19:43:09.224756 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224741    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wjx79\""
May 28 19:43:09.224993 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224850    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 19:43:09.224993 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.224917    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dkhnm\""
May 28 19:43:09.226554 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.226536    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.226662 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.226604    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:09.226960 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.226942    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 19:43:09.227019 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.226981    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 19:43:09.227148 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.227090    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g79r7\""
May 28 19:43:09.228823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.228805    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.231082 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.231061    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 19:43:09.231313 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.231295    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 19:43:09.231439 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.231422    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gqqgl\""
May 28 19:43:09.231523 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.231435    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.231635 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.231619    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 19:43:09.233713 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.233691    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 19:43:09.234067 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.233881    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gd7wp\""
May 28 19:43:09.234067 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.233936    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 19:43:09.234067 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.233982    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 19:43:09.235904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.234648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-multus-certs\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.235904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.234981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-tmp-dir\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.235904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.235032    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-ovn\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.235904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.235064    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/080ace10-d27e-47de-a72c-67815583d217-ovn-node-metrics-cert\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.235904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.235810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysctl-conf\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.235904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.235849    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-sys\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.235912    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.235949    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-var-lib-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236007    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/097c5ace-0c7c-495a-8e67-c9222b086068-agent-certs\") pod \"konnectivity-agent-c5fpz\" (UID: \"097c5ace-0c7c-495a-8e67-c9222b086068\") " pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-kubelet\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236092    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-systemd-units\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-cni-bin\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236177    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.236512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236222    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-conf-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.236512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236261    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p6wf\" (UniqueName: \"kubernetes.io/projected/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-kube-api-access-4p6wf\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.236512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236316    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236368    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-system-cni-dir\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.236512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236399    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-system-cni-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.236512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236434    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v7th\" (UniqueName: \"kubernetes.io/projected/080ace10-d27e-47de-a72c-67815583d217-kube-api-access-9v7th\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236512    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-systemd\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236571    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-lib-modules\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236614    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-host\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236668    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-ovnkube-script-lib\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236693    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/097c5ace-0c7c-495a-8e67-c9222b086068-konnectivity-ca\") pod \"konnectivity-agent-c5fpz\" (UID: \"097c5ace-0c7c-495a-8e67-c9222b086068\") " pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysconfig\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.236774 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236744    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-os-release\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236782    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-node-log\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236809    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236831    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysctl-d\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236861    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236913    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b81ecf59-2f24-43f5-a413-87f270208d8d-cni-binary-copy\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236950    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-cni-bin\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.236991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-etc-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-tuned\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.237070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237063    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-hostroot\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237095    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e87b2fb-50e4-450d-a1b1-4005da4611ec-tmp\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237121    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-cnibin\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-cni-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237179    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-netns\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237203    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-daemon-config\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237233    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-etc-kubernetes\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237262    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-env-overrides\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237289    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-systemd\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237343    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3305cb10-546e-48e3-ac9c-e012d6a0a326-host-slash\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237381    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-var-lib-kubelet\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237458    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.237537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237516    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-log-socket\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237545    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-cni-netd\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237574    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-kubernetes\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237685    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5ngq\" (UniqueName: \"kubernetes.io/projected/b81ecf59-2f24-43f5-a413-87f270208d8d-kube-api-access-g5ngq\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237723    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-slash\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j28fb\" (UniqueName: \"kubernetes.io/projected/3305cb10-546e-48e3-ac9c-e012d6a0a326-kube-api-access-j28fb\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237784    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-ovnkube-config\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237869    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-modprobe-d\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237899    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-495dq\" (UniqueName: \"kubernetes.io/projected/3e87b2fb-50e4-450d-a1b1-4005da4611ec-kube-api-access-495dq\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237944    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.237977    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-cnibin\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238007    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-hosts-file\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.238050 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-run-netns\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238064    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-k8s-cni-cncf-io\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238095    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-cni-multus\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-kubelet\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238166    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3305cb10-546e-48e3-ac9c-e012d6a0a326-iptables-alerter-script\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238233    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-run\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-os-release\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238324    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44ftt\" (UniqueName: \"kubernetes.io/projected/615454c2-0260-4ded-a724-210769c2ebfc-kube-api-access-44ftt\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.238686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.238373    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-socket-dir-parent\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.268305 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.268264    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 19:38:08 +0000 UTC" deadline="2027-11-06 07:41:50.695149142 +0000 UTC"
May 28 19:43:09.268305 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.268303    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12635h58m41.426849968s"
May 28 19:43:09.324485 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.324423    2568 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 19:43:09.338628 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338594    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338673    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9v7th\" (UniqueName: \"kubernetes.io/projected/080ace10-d27e-47de-a72c-67815583d217-kube-api-access-9v7th\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338706    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-systemd\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338732    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-lib-modules\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338757    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-host\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338790    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338794    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-systemd\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.338823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338817    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-host\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338842    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c67fw\" (UniqueName: \"kubernetes.io/projected/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-kube-api-access-c67fw\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338853    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-host\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338870    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-ovnkube-script-lib\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338872    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-lib-modules\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338910    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/097c5ace-0c7c-495a-8e67-c9222b086068-konnectivity-ca\") pod \"konnectivity-agent-c5fpz\" (UID: \"097c5ace-0c7c-495a-8e67-c9222b086068\") " pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338933    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysconfig\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338961    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-os-release\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.338992    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-node-log\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339013    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339040    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysctl-d\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339057    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339077    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b81ecf59-2f24-43f5-a413-87f270208d8d-cni-binary-copy\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339121    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-cni-bin\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339143    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-socket-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339145    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-os-release\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339158    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-device-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.339174 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339175    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-etc-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339346    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-cni-bin\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339379    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-tuned\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339411    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-hostroot\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339436    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e87b2fb-50e4-450d-a1b1-4005da4611ec-tmp\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339446    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysctl-d\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339546    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-cnibin\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339575    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-cni-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339605    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-netns\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339631    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-daemon-config\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339641    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-hostroot\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339605    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-ovnkube-script-lib\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339698    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysconfig\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339759    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339780    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-cni-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339822    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-etc-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339857    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-etc-kubernetes\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339876    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/097c5ace-0c7c-495a-8e67-c9222b086068-konnectivity-ca\") pod \"konnectivity-agent-c5fpz\" (UID: \"097c5ace-0c7c-495a-8e67-c9222b086068\") " pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.339942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339886    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-sys-fs\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339894    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b81ecf59-2f24-43f5-a413-87f270208d8d-cni-binary-copy\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339930    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-node-log\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339939    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-etc-kubernetes\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339967    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339973    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339989    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-netns\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.339969    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-cnibin\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340012    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-env-overrides\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340043    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-systemd\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340110    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3305cb10-546e-48e3-ac9c-e012d6a0a326-host-slash\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340115    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-daemon-config\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340140    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-var-lib-kubelet\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340190    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-serviceca\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340208    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340231    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-log-socket\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340248    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-cni-netd\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340285    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-kubernetes\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.340484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340301    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g5ngq\" (UniqueName: \"kubernetes.io/projected/b81ecf59-2f24-43f5-a413-87f270208d8d-kube-api-access-g5ngq\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340320    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-registration-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340337    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-slash\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340352    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j28fb\" (UniqueName: \"kubernetes.io/projected/3305cb10-546e-48e3-ac9c-e012d6a0a326-kube-api-access-j28fb\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340374    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340395    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-ovnkube-config\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340412    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-modprobe-d\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340494    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-495dq\" (UniqueName: \"kubernetes.io/projected/3e87b2fb-50e4-450d-a1b1-4005da4611ec-kube-api-access-495dq\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340524    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340542    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-log-socket\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340550    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-cnibin\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340574    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-hosts-file\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340612    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-run-netns\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340638    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-modprobe-d\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340642    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-k8s-cni-cncf-io\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340666    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-env-overrides\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340674    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-cni-netd\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.340980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340667    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-cni-multus\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340703    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-cni-multus\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340711    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-systemd\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340715    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-kubelet\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340746    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3305cb10-546e-48e3-ac9c-e012d6a0a326-iptables-alerter-script\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340753    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3305cb10-546e-48e3-ac9c-e012d6a0a326-host-slash\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340777    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-run\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340782    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-var-lib-kubelet\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340804    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-os-release\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340830    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-44ftt\" (UniqueName: \"kubernetes.io/projected/615454c2-0260-4ded-a724-210769c2ebfc-kube-api-access-44ftt\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340873    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcj5s\" (UniqueName: \"kubernetes.io/projected/e7082b38-7d27-4b14-abb0-0eeee423e2ac-kube-api-access-wcj5s\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340903    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-socket-dir-parent\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340930    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-multus-certs\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.340748    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-kubernetes\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341024    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-tmp-dir\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341052    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-ovn\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341078    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/080ace10-d27e-47de-a72c-67815583d217-ovn-node-metrics-cert\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.341431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341107    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysctl-conf\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341132    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-sys\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341161    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341188    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-var-lib-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341209    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341215    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-slash\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341217    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/097c5ace-0c7c-495a-8e67-c9222b086068-agent-certs\") pod \"konnectivity-agent-c5fpz\" (UID: \"097c5ace-0c7c-495a-8e67-c9222b086068\") " pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341261    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-kubelet\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341268    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-cnibin\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341314    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341333    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-hosts-file\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341349    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-var-lib-kubelet\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341367    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-run-netns\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341404    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-k8s-cni-cncf-io\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341424    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-sysctl-conf\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341443    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-run-ovn\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341459    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-host-run-multus-certs\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341486    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-run\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.342210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341526    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-socket-dir-parent\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341583    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-os-release\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.341913    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3305cb10-546e-48e3-ac9c-e012d6a0a326-iptables-alerter-script\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342231    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-tmp-dir\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342338    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/080ace10-d27e-47de-a72c-67815583d217-ovnkube-config\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342386    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-systemd-units\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342413    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-cni-bin\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342446    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4hsx\" (UniqueName: \"kubernetes.io/projected/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-kube-api-access-t4hsx\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342495    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342523    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-conf-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342568    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6wf\" (UniqueName: \"kubernetes.io/projected/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-kube-api-access-4p6wf\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342587    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342603    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-system-cni-dir\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342630    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-system-cni-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.343107 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.342658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.344153 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344125    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-multus-conf-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.344263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344169    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-kubelet\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.344263 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344198    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e87b2fb-50e4-450d-a1b1-4005da4611ec-sys\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.344361 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344280    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.344446 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344426    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-var-lib-openvswitch\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.344550 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344532    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-systemd-units\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.344606 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344580    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-cni-bin\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.345019 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.344988    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e87b2fb-50e4-450d-a1b1-4005da4611ec-tmp\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.345227 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.345201    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/615454c2-0260-4ded-a724-210769c2ebfc-cni-binary-copy\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.346185 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.346163    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3e87b2fb-50e4-450d-a1b1-4005da4611ec-etc-tuned\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.346300 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.346225    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/080ace10-d27e-47de-a72c-67815583d217-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.346300 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.346272    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/615454c2-0260-4ded-a724-210769c2ebfc-system-cni-dir\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.347241 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.347022    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/080ace10-d27e-47de-a72c-67815583d217-ovn-node-metrics-cert\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.347889 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.347819    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b81ecf59-2f24-43f5-a413-87f270208d8d-system-cni-dir\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.347986 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.347943    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:09.347986 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.347966    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:09.347986 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.347980    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:09.348094 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.348062    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:09.848040144 +0000 UTC m=+3.135381030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:09.348552 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.348482    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/097c5ace-0c7c-495a-8e67-c9222b086068-agent-certs\") pod \"konnectivity-agent-c5fpz\" (UID: \"097c5ace-0c7c-495a-8e67-c9222b086068\") " pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.352140 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.352105    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5ngq\" (UniqueName: \"kubernetes.io/projected/b81ecf59-2f24-43f5-a413-87f270208d8d-kube-api-access-g5ngq\") pod \"multus-ngpzm\" (UID: \"b81ecf59-2f24-43f5-a413-87f270208d8d\") " pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.352315 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.352162    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-495dq\" (UniqueName: \"kubernetes.io/projected/3e87b2fb-50e4-450d-a1b1-4005da4611ec-kube-api-access-495dq\") pod \"tuned-gmh25\" (UID: \"3e87b2fb-50e4-450d-a1b1-4005da4611ec\") " pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.352431 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.352411    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v7th\" (UniqueName: \"kubernetes.io/projected/080ace10-d27e-47de-a72c-67815583d217-kube-api-access-9v7th\") pod \"ovnkube-node-rxvsp\" (UID: \"080ace10-d27e-47de-a72c-67815583d217\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.354172 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.353954    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44ftt\" (UniqueName: \"kubernetes.io/projected/615454c2-0260-4ded-a724-210769c2ebfc-kube-api-access-44ftt\") pod \"multus-additional-cni-plugins-j7t6m\" (UID: \"615454c2-0260-4ded-a724-210769c2ebfc\") " pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.355209 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.355187    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p6wf\" (UniqueName: \"kubernetes.io/projected/054a4749-b39d-4f52-b4cc-cc15a9a93bf8-kube-api-access-4p6wf\") pod \"node-resolver-rflwq\" (UID: \"054a4749-b39d-4f52-b4cc-cc15a9a93bf8\") " pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.355849 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.355832    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j28fb\" (UniqueName: \"kubernetes.io/projected/3305cb10-546e-48e3-ac9c-e012d6a0a326-kube-api-access-j28fb\") pod \"iptables-alerter-btw6b\" (UID: \"3305cb10-546e-48e3-ac9c-e012d6a0a326\") " pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.444521 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444488    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wcj5s\" (UniqueName: \"kubernetes.io/projected/e7082b38-7d27-4b14-abb0-0eeee423e2ac-kube-api-access-wcj5s\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444678 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444538    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t4hsx\" (UniqueName: \"kubernetes.io/projected/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-kube-api-access-t4hsx\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.444749 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444685    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444749 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444715    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444749 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444745    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444769    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-host\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444781    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444791    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c67fw\" (UniqueName: \"kubernetes.io/projected/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-kube-api-access-c67fw\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444812    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444850    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-socket-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444879    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-device-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444896    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-host\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.444902    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:09.444927 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444913    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-sys-fs\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444941    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-serviceca\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.444971    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:43:09.944955592 +0000 UTC m=+3.232296487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.444977    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-device-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.445000    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-registration-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.445031    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-sys-fs\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.445034    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-socket-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.445235 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.445065    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7082b38-7d27-4b14-abb0-0eeee423e2ac-registration-dir\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.445504 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.445301    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-serviceca\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.454456 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.454429    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4hsx\" (UniqueName: \"kubernetes.io/projected/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-kube-api-access-t4hsx\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.454980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.454943    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcj5s\" (UniqueName: \"kubernetes.io/projected/e7082b38-7d27-4b14-abb0-0eeee423e2ac-kube-api-access-wcj5s\") pod \"aws-ebs-csi-driver-node-c95lq\" (UID: \"e7082b38-7d27-4b14-abb0-0eeee423e2ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.455279 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.455255    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c67fw\" (UniqueName: \"kubernetes.io/projected/bd7826c6-6b85-45d0-9a3a-c1b4fca63208-kube-api-access-c67fw\") pod \"node-ca-hl88v\" (UID: \"bd7826c6-6b85-45d0-9a3a-c1b4fca63208\") " pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.518316 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.518223    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:09.523251 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.523222    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j7t6m"
May 28 19:43:09.533119 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.533090    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ngpzm"
May 28 19:43:09.540869 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.540834    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-btw6b"
May 28 19:43:09.546402 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.546376    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:09.553162 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.553127    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gmh25"
May 28 19:43:09.559943 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.559854    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rflwq"
May 28 19:43:09.565558 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.565532    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq"
May 28 19:43:09.571141 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.571115    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hl88v"
May 28 19:43:09.947863 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.947768    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:09.947863 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:09.947824    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:09.948089 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.947930    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:09.948089 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.947951    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:09.948089 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.947975    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:09.948089 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.947988    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:09.948089 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.947992    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:43:10.947975611 +0000 UTC m=+4.235316494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:09.948089 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:09.948039    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:10.948022754 +0000 UTC m=+4.235363640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:10.027136 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.027101    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097c5ace_0c7c_495a_8e67_c9222b086068.slice/crio-3fd6dd06b72e56ff6407d8153a7d2b25f849e6649b0555e37f8cf51944b8847d WatchSource:0}: Error finding container 3fd6dd06b72e56ff6407d8153a7d2b25f849e6649b0555e37f8cf51944b8847d: Status 404 returned error can't find the container with id 3fd6dd06b72e56ff6407d8153a7d2b25f849e6649b0555e37f8cf51944b8847d
May 28 19:43:10.027930 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.027902    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615454c2_0260_4ded_a724_210769c2ebfc.slice/crio-0f12441540d609d7202653e2ffff92cb901df1084d516d7d3923e2f6dc85c5c6 WatchSource:0}: Error finding container 0f12441540d609d7202653e2ffff92cb901df1084d516d7d3923e2f6dc85c5c6: Status 404 returned error can't find the container with id 0f12441540d609d7202653e2ffff92cb901df1084d516d7d3923e2f6dc85c5c6
May 28 19:43:10.028849 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.028761    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3305cb10_546e_48e3_ac9c_e012d6a0a326.slice/crio-f41436847a0718a51ca05c953e720c1c1f978e149694f8c01ed172b45581e731 WatchSource:0}: Error finding container f41436847a0718a51ca05c953e720c1c1f978e149694f8c01ed172b45581e731: Status 404 returned error can't find the container with id f41436847a0718a51ca05c953e720c1c1f978e149694f8c01ed172b45581e731
May 28 19:43:10.030628 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.030538    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod054a4749_b39d_4f52_b4cc_cc15a9a93bf8.slice/crio-83d79b557df90dc9f62ae54f35429e643e81f7284ab3e0ba0b283db8c670abb3 WatchSource:0}: Error finding container 83d79b557df90dc9f62ae54f35429e643e81f7284ab3e0ba0b283db8c670abb3: Status 404 returned error can't find the container with id 83d79b557df90dc9f62ae54f35429e643e81f7284ab3e0ba0b283db8c670abb3
May 28 19:43:10.031759 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.031720    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81ecf59_2f24_43f5_a413_87f270208d8d.slice/crio-eaa131873e414ff97d17d9db65db6a29f03f6b530f4210e3b9558f615d976f14 WatchSource:0}: Error finding container eaa131873e414ff97d17d9db65db6a29f03f6b530f4210e3b9558f615d976f14: Status 404 returned error can't find the container with id eaa131873e414ff97d17d9db65db6a29f03f6b530f4210e3b9558f615d976f14
May 28 19:43:10.037105 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.037080    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7082b38_7d27_4b14_abb0_0eeee423e2ac.slice/crio-1f9b32683e4484d5b0f1908d6201a10f8ba17d7017087f8080769850df60454e WatchSource:0}: Error finding container 1f9b32683e4484d5b0f1908d6201a10f8ba17d7017087f8080769850df60454e: Status 404 returned error can't find the container with id 1f9b32683e4484d5b0f1908d6201a10f8ba17d7017087f8080769850df60454e
May 28 19:43:10.057927 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.057897    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080ace10_d27e_47de_a72c_67815583d217.slice/crio-360d4b0ee9953291bb6ae4c9690a25f34ca45bea060b7946d3aca6ab9c8e6759 WatchSource:0}: Error finding container 360d4b0ee9953291bb6ae4c9690a25f34ca45bea060b7946d3aca6ab9c8e6759: Status 404 returned error can't find the container with id 360d4b0ee9953291bb6ae4c9690a25f34ca45bea060b7946d3aca6ab9c8e6759
May 28 19:43:10.059417 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.059379    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e87b2fb_50e4_450d_a1b1_4005da4611ec.slice/crio-76427dacabef1040209f61324c25b11134e58f7bd0772787362dbba67513baee WatchSource:0}: Error finding container 76427dacabef1040209f61324c25b11134e58f7bd0772787362dbba67513baee: Status 404 returned error can't find the container with id 76427dacabef1040209f61324c25b11134e58f7bd0772787362dbba67513baee
May 28 19:43:10.060229 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:10.060200    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7826c6_6b85_45d0_9a3a_c1b4fca63208.slice/crio-27f93acb69c72a8d2a69eb3589a8732151d2026ccfcefb5987041b27bef1b27a WatchSource:0}: Error finding container 27f93acb69c72a8d2a69eb3589a8732151d2026ccfcefb5987041b27bef1b27a: Status 404 returned error can't find the container with id 27f93acb69c72a8d2a69eb3589a8732151d2026ccfcefb5987041b27bef1b27a
May 28 19:43:10.269025 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.268913    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 19:38:08 +0000 UTC" deadline="2028-02-12 17:27:19.431432972 +0000 UTC"
May 28 19:43:10.269025 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.268948    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14997h44m9.162487923s"
May 28 19:43:10.329322 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.329274    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:10.329489 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.329392    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:10.337215 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.337173    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal" event={"ID":"f0dac381d94558e70808c6353a7e9fc8","Type":"ContainerStarted","Data":"a9c72a3bdb49b4c94031d934f537e142fb1031a599cb062830712384877cae38"}
May 28 19:43:10.338369 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.338330    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"360d4b0ee9953291bb6ae4c9690a25f34ca45bea060b7946d3aca6ab9c8e6759"}
May 28 19:43:10.339271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.339240    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-btw6b" event={"ID":"3305cb10-546e-48e3-ac9c-e012d6a0a326","Type":"ContainerStarted","Data":"f41436847a0718a51ca05c953e720c1c1f978e149694f8c01ed172b45581e731"}
May 28 19:43:10.340130 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.340109    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5fpz" event={"ID":"097c5ace-0c7c-495a-8e67-c9222b086068","Type":"ContainerStarted","Data":"3fd6dd06b72e56ff6407d8153a7d2b25f849e6649b0555e37f8cf51944b8847d"}
May 28 19:43:10.341091 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.341074    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerStarted","Data":"0f12441540d609d7202653e2ffff92cb901df1084d516d7d3923e2f6dc85c5c6"}
May 28 19:43:10.342028 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.342011    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hl88v" event={"ID":"bd7826c6-6b85-45d0-9a3a-c1b4fca63208","Type":"ContainerStarted","Data":"27f93acb69c72a8d2a69eb3589a8732151d2026ccfcefb5987041b27bef1b27a"}
May 28 19:43:10.342847 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.342830    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gmh25" event={"ID":"3e87b2fb-50e4-450d-a1b1-4005da4611ec","Type":"ContainerStarted","Data":"76427dacabef1040209f61324c25b11134e58f7bd0772787362dbba67513baee"}
May 28 19:43:10.343899 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.343875    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq" event={"ID":"e7082b38-7d27-4b14-abb0-0eeee423e2ac","Type":"ContainerStarted","Data":"1f9b32683e4484d5b0f1908d6201a10f8ba17d7017087f8080769850df60454e"}
May 28 19:43:10.344954 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.344937    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ngpzm" event={"ID":"b81ecf59-2f24-43f5-a413-87f270208d8d","Type":"ContainerStarted","Data":"eaa131873e414ff97d17d9db65db6a29f03f6b530f4210e3b9558f615d976f14"}
May 28 19:43:10.345914 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.345897    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rflwq" event={"ID":"054a4749-b39d-4f52-b4cc-cc15a9a93bf8","Type":"ContainerStarted","Data":"83d79b557df90dc9f62ae54f35429e643e81f7284ab3e0ba0b283db8c670abb3"}
May 28 19:43:10.352371 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.352311    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-191.ec2.internal" podStartSLOduration=2.352294915 podStartE2EDuration="2.352294915s" podCreationTimestamp="2026-05-28 19:43:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 19:43:10.35201559 +0000 UTC m=+3.639356494" watchObservedRunningTime="2026-05-28 19:43:10.352294915 +0000 UTC m=+3.639635820"
May 28 19:43:10.956941 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.956896    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:10.957139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:10.956967    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:10.957139 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.957113    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:10.957261 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.957177    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:43:12.957158492 +0000 UTC m=+6.244499380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:10.958145 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.957605    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:10.958145 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.957629    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:10.958145 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.957641    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:10.958145 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:10.957695    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:12.957676426 +0000 UTC m=+6.245017330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:11.330210 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:11.330121    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:11.330761 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:11.330260    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:11.368034 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:11.367991    2568 generic.go:358] "Generic (PLEG): container finished" podID="b36985eef541dcd08c713a8cd1eb1e11" containerID="640c3a3675a65e62a0a678fc622e04648862d17f430cfa310e52eb0ae1e17d60" exitCode=0
May 28 19:43:11.369047 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:11.369018    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal" event={"ID":"b36985eef541dcd08c713a8cd1eb1e11","Type":"ContainerDied","Data":"640c3a3675a65e62a0a678fc622e04648862d17f430cfa310e52eb0ae1e17d60"}
May 28 19:43:12.330351 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:12.330314    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:12.330893 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.330489    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:12.399373 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:12.399329    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal" event={"ID":"b36985eef541dcd08c713a8cd1eb1e11","Type":"ContainerStarted","Data":"f794763dd6af51f955be7f9a8cc3bfcfd6010b508d221093f4118128662a6bee"}
May 28 19:43:12.415716 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:12.415643    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-191.ec2.internal" podStartSLOduration=4.415624957 podStartE2EDuration="4.415624957s" podCreationTimestamp="2026-05-28 19:43:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 19:43:12.414389996 +0000 UTC m=+5.701730903" watchObservedRunningTime="2026-05-28 19:43:12.415624957 +0000 UTC m=+5.702965865"
May 28 19:43:12.974008 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:12.973967    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:12.974185 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:12.974052    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:12.974269 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.974214    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:12.974269 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.974234    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:12.974269 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.974247    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:12.974396 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.974310    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:16.974289835 +0000 UTC m=+10.261630740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:12.974609 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.974569    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:12.974755 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:12.974653    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:43:16.974635726 +0000 UTC m=+10.261976607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:13.329998 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:13.329899    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:13.330170 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:13.330043    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:14.330323 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:14.329805    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:14.330323 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:14.329965    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:15.332360 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:15.332320    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:15.332837 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:15.332460    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:16.221879 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.221833    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-px2n9"]
May 28 19:43:16.226525 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.226037    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.226525 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:16.226135    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:16.304717 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.304676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11c4e6c3-c807-4c58-bf0a-551c264cb561-kubelet-config\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.304884 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.304755    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.304884 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.304786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11c4e6c3-c807-4c58-bf0a-551c264cb561-dbus\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.329805 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.329767    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:16.329977 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:16.329898    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:16.405721 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.405681    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11c4e6c3-c807-4c58-bf0a-551c264cb561-kubelet-config\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.406198 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.405764    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.406198 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.405798    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11c4e6c3-c807-4c58-bf0a-551c264cb561-dbus\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.406198 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.405967    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/11c4e6c3-c807-4c58-bf0a-551c264cb561-dbus\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.406198 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.406027    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/11c4e6c3-c807-4c58-bf0a-551c264cb561-kubelet-config\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.406198 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:16.406121    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 19:43:16.406198 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:16.406171    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret podName:11c4e6c3-c807-4c58-bf0a-551c264cb561 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:16.906155838 +0000 UTC m=+10.193496719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret") pod "global-pull-secret-syncer-px2n9" (UID: "11c4e6c3-c807-4c58-bf0a-551c264cb561") : object "kube-system"/"original-pull-secret" not registered
May 28 19:43:16.910765 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:16.910728    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:16.910963 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:16.910868    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 19:43:16.910963 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:16.910936    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret podName:11c4e6c3-c807-4c58-bf0a-551c264cb561 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:17.910917534 +0000 UTC m=+11.198258416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret") pod "global-pull-secret-syncer-px2n9" (UID: "11c4e6c3-c807-4c58-bf0a-551c264cb561") : object "kube-system"/"original-pull-secret" not registered
May 28 19:43:17.011081 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:17.011042    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:17.011282 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:17.011135    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:17.011282 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.011271    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:17.011372 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.011286    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:17.011372 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.011295    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:17.011372 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.011336    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:25.011324354 +0000 UTC m=+18.298665235 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:17.011642 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.011615    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:17.011768 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.011676    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:43:25.011661439 +0000 UTC m=+18.299002321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:17.330759 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:17.330678    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:17.330898 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.330796    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:17.920406 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:17.920366    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:17.920955 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.920547    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 19:43:17.920955 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:17.920636    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret podName:11c4e6c3-c807-4c58-bf0a-551c264cb561 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:19.920611654 +0000 UTC m=+13.207952551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret") pod "global-pull-secret-syncer-px2n9" (UID: "11c4e6c3-c807-4c58-bf0a-551c264cb561") : object "kube-system"/"original-pull-secret" not registered
May 28 19:43:18.330354 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:18.330259    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:18.330354 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:18.330313    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:18.330636 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:18.330488    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:18.330695 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:18.330621    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:19.332737 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:19.331841    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:19.332737 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:19.331979    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:19.937325 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:19.937282    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:19.937545 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:19.937461    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 19:43:19.937606 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:19.937572    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret podName:11c4e6c3-c807-4c58-bf0a-551c264cb561 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:23.937549491 +0000 UTC m=+17.224890403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret") pod "global-pull-secret-syncer-px2n9" (UID: "11c4e6c3-c807-4c58-bf0a-551c264cb561") : object "kube-system"/"original-pull-secret" not registered
May 28 19:43:20.329854 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:20.329824    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:20.330038 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:20.329832    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:20.330038 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:20.329953    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:20.330142 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:20.330078    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:21.330402 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:21.330359    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:21.330890 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:21.330511    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:22.330240 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:22.330201    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:22.330434 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:22.330206    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:22.330434 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:22.330344    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:22.330816 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:22.330425    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:23.329572 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:23.329530    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:23.329754 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:23.329645    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:23.965804 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:23.965708    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:23.966182 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:23.965836    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 19:43:23.966182 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:23.965894    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret podName:11c4e6c3-c807-4c58-bf0a-551c264cb561 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:31.965881301 +0000 UTC m=+25.253222182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret") pod "global-pull-secret-syncer-px2n9" (UID: "11c4e6c3-c807-4c58-bf0a-551c264cb561") : object "kube-system"/"original-pull-secret" not registered
May 28 19:43:24.329355 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:24.329269    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:24.329563 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:24.329274    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:24.329563 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:24.329392    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:24.329563 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:24.329493    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:25.073203 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:25.073161    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:25.073614 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:25.073215    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:25.073614 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.073336    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:25.073614 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.073389    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:43:41.073374497 +0000 UTC m=+34.360715379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:25.073614 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.073601    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:25.073756 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.073623    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:25.073756 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.073636    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:25.073756 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.073685    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:41.073670324 +0000 UTC m=+34.361011206 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:25.329823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:25.329732    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:25.329985 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:25.329884    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:26.329828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:26.329784    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:26.330281 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:26.329784    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:26.330281 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:26.329952    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:26.330281 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:26.329990    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:27.330091 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:27.330049    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:27.330506 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:27.330168    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:28.329334 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:28.329298    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:28.329528 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:28.329298    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:28.329528 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:28.329420    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:28.329528 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:28.329518    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:29.329945 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.329672    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:29.330927 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:29.330057    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:29.431874 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.431843    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:43:29.432153 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432128    2568 generic.go:358] "Generic (PLEG): container finished" podID="080ace10-d27e-47de-a72c-67815583d217" containerID="a1f7b6944a94f24884e96021bb624e851a8043a0d69072e6914f907c651c5e35" exitCode=1
May 28 19:43:29.432224 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432198    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"5d389a17ef7bee00344d7f8bd723cfe4e259f554d502598ba8792551f1dd13eb"}
May 28 19:43:29.432283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432241    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"96149410d95d441378d01d794ea70603e35263627dfd06fb9e3c978910ddecf0"}
May 28 19:43:29.432283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432255    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"3cf2528e9cd5b53bbd6cb56502495a4950036436c009d572d5f83d51f7afe99e"}
May 28 19:43:29.432283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432267    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"3d035e92b6a0055e947ca5964767c01934892e8a8452ad27cc163ea90c2e5b8d"}
May 28 19:43:29.432283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432278    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerDied","Data":"a1f7b6944a94f24884e96021bb624e851a8043a0d69072e6914f907c651c5e35"}
May 28 19:43:29.432462 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.432295    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"6d1f3c6826021a1739377e03e9f00cdee536e6e893107093398618096e61d004"}
May 28 19:43:29.433979 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.433952    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c5fpz" event={"ID":"097c5ace-0c7c-495a-8e67-c9222b086068","Type":"ContainerStarted","Data":"5cb44f5e67fd15369439851988845ffd3e69f49aabeb035d17b6d8265706c9eb"}
May 28 19:43:29.435417 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.435391    2568 generic.go:358] "Generic (PLEG): container finished" podID="615454c2-0260-4ded-a724-210769c2ebfc" containerID="bba02bc2273f8b80052261afb8ce81edcfac1b521d0c1fbb4426b4dea8e975bc" exitCode=0
May 28 19:43:29.435558 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.435460    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerDied","Data":"bba02bc2273f8b80052261afb8ce81edcfac1b521d0c1fbb4426b4dea8e975bc"}
May 28 19:43:29.436806 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.436764    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hl88v" event={"ID":"bd7826c6-6b85-45d0-9a3a-c1b4fca63208","Type":"ContainerStarted","Data":"b3ee4a5fbb41bb47a78651410d4d3a364a77bea41e197e6a3759d31f5e2654f7"}
May 28 19:43:29.438165 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.438139    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gmh25" event={"ID":"3e87b2fb-50e4-450d-a1b1-4005da4611ec","Type":"ContainerStarted","Data":"d4acf7e7217b9364b405b5985f5724eed78d14fde833b2df6a5dc50177131d93"}
May 28 19:43:29.439685 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.439664    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq" event={"ID":"e7082b38-7d27-4b14-abb0-0eeee423e2ac","Type":"ContainerStarted","Data":"4f54d1daff2457bb0483fcce167e27d8decde0a21beda432ab1b0094de7ea5b4"}
May 28 19:43:29.440933 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.440915    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ngpzm" event={"ID":"b81ecf59-2f24-43f5-a413-87f270208d8d","Type":"ContainerStarted","Data":"e82fc658b86d154df8c127f5618162845895f0151c8600acf807d10734d3010a"}
May 28 19:43:29.442164 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.442146    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rflwq" event={"ID":"054a4749-b39d-4f52-b4cc-cc15a9a93bf8","Type":"ContainerStarted","Data":"b39b8b403b836a348651b00d7d0b8bc3b889299e3bbb1ea0253f20bca9a0f296"}
May 28 19:43:29.462116 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.462012    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c5fpz" podStartSLOduration=4.052062346 podStartE2EDuration="22.461997324s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.029097427 +0000 UTC m=+3.316438310" lastFinishedPulling="2026-05-28 19:43:28.439032406 +0000 UTC m=+21.726373288" observedRunningTime="2026-05-28 19:43:29.461343363 +0000 UTC m=+22.748684268" watchObservedRunningTime="2026-05-28 19:43:29.461997324 +0000 UTC m=+22.749338227"
May 28 19:43:29.478283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.478233    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-ngpzm" podStartSLOduration=4.037058516 podStartE2EDuration="22.478217992s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.035035911 +0000 UTC m=+3.322376794" lastFinishedPulling="2026-05-28 19:43:28.476195388 +0000 UTC m=+21.763536270" observedRunningTime="2026-05-28 19:43:29.477830361 +0000 UTC m=+22.765171265" watchObservedRunningTime="2026-05-28 19:43:29.478217992 +0000 UTC m=+22.765558897"
May 28 19:43:29.516141 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.516092    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hl88v" podStartSLOduration=12.28494457 podStartE2EDuration="22.51607959s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.062645369 +0000 UTC m=+3.349986258" lastFinishedPulling="2026-05-28 19:43:20.293780386 +0000 UTC m=+13.581121278" observedRunningTime="2026-05-28 19:43:29.491669831 +0000 UTC m=+22.779010736" watchObservedRunningTime="2026-05-28 19:43:29.51607959 +0000 UTC m=+22.803420494"
May 28 19:43:29.532461 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.532397    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gmh25" podStartSLOduration=4.155916405 podStartE2EDuration="22.532376207s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.062552653 +0000 UTC m=+3.349893535" lastFinishedPulling="2026-05-28 19:43:28.439012453 +0000 UTC m=+21.726353337" observedRunningTime="2026-05-28 19:43:29.531588686 +0000 UTC m=+22.818929591" watchObservedRunningTime="2026-05-28 19:43:29.532376207 +0000 UTC m=+22.819717113"
May 28 19:43:29.546899 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.546845    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-rflwq" podStartSLOduration=4.139960372 podStartE2EDuration="22.5468296s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.032276069 +0000 UTC m=+3.319616950" lastFinishedPulling="2026-05-28 19:43:28.439145296 +0000 UTC m=+21.726486178" observedRunningTime="2026-05-28 19:43:29.546230309 +0000 UTC m=+22.833571213" watchObservedRunningTime="2026-05-28 19:43:29.5468296 +0000 UTC m=+22.834170503"
May 28 19:43:29.806118 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.806088    2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:29.806775 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:29.806759    2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:30.187980 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.187796    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 19:43:30.302110 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.301992    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T19:43:30.187971163Z","UUID":"27a8bf2c-ccf1-498b-bcb9-020b545e0134","Handler":null,"Name":"","Endpoint":""}
May 28 19:43:30.305093 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.305063    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 19:43:30.305093 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.305099    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 19:43:30.329295 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.329220    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:30.329295 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.329287    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:30.329553 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:30.329392    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:30.329553 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:30.329532    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:30.445757 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.445719    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-btw6b" event={"ID":"3305cb10-546e-48e3-ac9c-e012d6a0a326","Type":"ContainerStarted","Data":"0053d9a2e921e1f1f0cf0214eff59f358a828f0eed0dcdbb7b54202cffde9315"}
May 28 19:43:30.447367 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.447335    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq" event={"ID":"e7082b38-7d27-4b14-abb0-0eeee423e2ac","Type":"ContainerStarted","Data":"9660b771fe178d9082dd04d0a6dbde2c64d9267fd0e58c4513e33fa1675174fa"}
May 28 19:43:30.448075 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.448054    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:30.448520 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.448504    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c5fpz"
May 28 19:43:30.460901 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:30.460857    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-btw6b" podStartSLOduration=5.052671742 podStartE2EDuration="23.460843093s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.031031524 +0000 UTC m=+3.318372406" lastFinishedPulling="2026-05-28 19:43:28.439202868 +0000 UTC m=+21.726543757" observedRunningTime="2026-05-28 19:43:30.460413974 +0000 UTC m=+23.747754912" watchObservedRunningTime="2026-05-28 19:43:30.460843093 +0000 UTC m=+23.748183997"
May 28 19:43:31.330214 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:31.330190    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:31.330353 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:31.330300    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:31.453070 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:31.453043    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:43:31.453646 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:31.453417    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"f3ecd3052e64f3d3e40a11688de73679e1a4cbaa554c4d3870e2c65f7012ea71"}
May 28 19:43:31.456850 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:31.456188    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq" event={"ID":"e7082b38-7d27-4b14-abb0-0eeee423e2ac","Type":"ContainerStarted","Data":"57a84588f090fe7d270640707172268e28dfcbc9b5a3589fb0c94d4c0ed0caa6"}
May 28 19:43:31.477680 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:31.477616    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c95lq" podStartSLOduration=3.265426718 podStartE2EDuration="24.477594615s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.056661894 +0000 UTC m=+3.344002776" lastFinishedPulling="2026-05-28 19:43:31.268829786 +0000 UTC m=+24.556170673" observedRunningTime="2026-05-28 19:43:31.47668797 +0000 UTC m=+24.764028874" watchObservedRunningTime="2026-05-28 19:43:31.477594615 +0000 UTC m=+24.764935521"
May 28 19:43:32.025071 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:32.025028    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:32.025251 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:32.025190    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
May 28 19:43:32.025294 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:32.025274    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret podName:11c4e6c3-c807-4c58-bf0a-551c264cb561 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:48.025251312 +0000 UTC m=+41.312592198 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret") pod "global-pull-secret-syncer-px2n9" (UID: "11c4e6c3-c807-4c58-bf0a-551c264cb561") : object "kube-system"/"original-pull-secret" not registered
May 28 19:43:32.329582 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:32.329487    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:32.329740 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:32.329495    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:32.329740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:32.329595    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:32.329740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:32.329671    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:33.330090 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:33.330017    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:33.330825 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:33.330189    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:34.329571 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.329537    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:34.329731 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.329575    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:34.329731 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:34.329650    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:34.329827 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:34.329803    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:34.464875 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.464775    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:43:34.465285 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.465135    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"19ce5010eb94abe13408709c8455e5d35501215df03be084388313b5ee2dd395"}
May 28 19:43:34.465559 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.465501    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:34.465559 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.465538    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:34.465750 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.465664    2568 scope.go:117] "RemoveContainer" containerID="a1f7b6944a94f24884e96021bb624e851a8043a0d69072e6914f907c651c5e35"
May 28 19:43:34.467264 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.467232    2568 generic.go:358] "Generic (PLEG): container finished" podID="615454c2-0260-4ded-a724-210769c2ebfc" containerID="1bef02467edf1e75278328cc1b92de8d270770e93fd451b11c5bdf33fe4000d8" exitCode=0
May 28 19:43:34.467377 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.467279    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerDied","Data":"1bef02467edf1e75278328cc1b92de8d270770e93fd451b11c5bdf33fe4000d8"}
May 28 19:43:34.483761 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.483733    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:34.483909 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:34.483823    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:35.329412 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.329374    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:35.329586 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:35.329504    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:35.473654 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.473564    2568 generic.go:358] "Generic (PLEG): container finished" podID="615454c2-0260-4ded-a724-210769c2ebfc" containerID="8b95910276534a08d2e9325c33a8bd93a25ef00bbe8ead519a16d0bf29326248" exitCode=0
May 28 19:43:35.474103 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.473680    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerDied","Data":"8b95910276534a08d2e9325c33a8bd93a25ef00bbe8ead519a16d0bf29326248"}
May 28 19:43:35.499990 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.499611    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:43:35.500157 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.500125    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" event={"ID":"080ace10-d27e-47de-a72c-67815583d217","Type":"ContainerStarted","Data":"ba5b0991d5e2fcaa58f77993f65d08d95142a8ddb60e3b632eb3ad52ecf3153d"}
May 28 19:43:35.503493 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.500362    2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 19:43:35.527215 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:35.527163    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp" podStartSLOduration=10.032980676 podStartE2EDuration="28.527149229s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.059701469 +0000 UTC m=+3.347042351" lastFinishedPulling="2026-05-28 19:43:28.553870019 +0000 UTC m=+21.841210904" observedRunningTime="2026-05-28 19:43:35.526679135 +0000 UTC m=+28.814020039" watchObservedRunningTime="2026-05-28 19:43:35.527149229 +0000 UTC m=+28.814490133"
May 28 19:43:36.152015 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.151838    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxzjd"]
May 28 19:43:36.152161 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.152129    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:36.152245 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:36.152227    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:36.155038 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.154914    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r9l4n"]
May 28 19:43:36.155114 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.155045    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:36.155170 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:36.155135    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:36.155607 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.155582    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-px2n9"]
May 28 19:43:36.155723 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.155703    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:36.155822 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:36.155799    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:36.504458 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.504359    2568 generic.go:358] "Generic (PLEG): container finished" podID="615454c2-0260-4ded-a724-210769c2ebfc" containerID="7d2c0e3b8f9573e2c4d5788d2e96343282aee435d94e63f9dfbeb148c9bcae38" exitCode=0
May 28 19:43:36.504458 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.504439    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerDied","Data":"7d2c0e3b8f9573e2c4d5788d2e96343282aee435d94e63f9dfbeb148c9bcae38"}
May 28 19:43:36.504941 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:36.504719    2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
May 28 19:43:37.171669 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:37.171632    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:43:37.330780 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:37.330264    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:37.330780 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:37.330385    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:37.330780 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:37.330535    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:37.330780 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:37.330380    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:38.330157 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:38.330121    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:38.330880 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:38.330237    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:39.330195 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:39.330159    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:39.330749 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:39.330301    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:39.330749 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:39.330352    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:39.330749 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:39.330486    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:40.329359 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:40.329301    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:40.329577 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:40.329457    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r9l4n" podUID="e9e44915-c783-4fb0-84e0-1b0c0d67dfd2"
May 28 19:43:41.096644 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.096435    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:41.096644 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.096519    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:41.096644 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.096625    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 19:43:41.097213 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.096650    2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 19:43:41.097213 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.096665    2568 projected.go:194] Error preparing data for projected volume kube-api-access-db9kx for pod openshift-network-diagnostics/network-check-target-r9l4n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:41.097213 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.096690    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:41.097213 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.096728    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx podName:e9e44915-c783-4fb0-84e0-1b0c0d67dfd2 nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.096714108 +0000 UTC m=+66.384054996 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-db9kx" (UniqueName: "kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx") pod "network-check-target-r9l4n" (UID: "e9e44915-c783-4fb0-84e0-1b0c0d67dfd2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 19:43:41.097213 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.096748    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs podName:0f40971c-dd1a-4403-a51b-bdd13d1fb3cc nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.096734295 +0000 UTC m=+66.384075177 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs") pod "network-metrics-daemon-wxzjd" (UID: "0f40971c-dd1a-4403-a51b-bdd13d1fb3cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 19:43:41.329718 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.329622    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:41.329874 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.329636    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:41.329874 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.329749    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-px2n9" podUID="11c4e6c3-c807-4c58-bf0a-551c264cb561"
May 28 19:43:41.329874 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.329840    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-wxzjd" podUID="0f40971c-dd1a-4403-a51b-bdd13d1fb3cc"
May 28 19:43:41.524201 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.524172    2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-135-191.ec2.internal" event="NodeReady"
May 28 19:43:41.524387 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.524307    2568 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 19:43:41.562317 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.562281    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"]
May 28 19:43:41.602464 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.602379    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-9vlfb"]
May 28 19:43:41.619782 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.619743    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"]
May 28 19:43:41.619966 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.619889    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:41.620344 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.620321    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.622792 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.622771    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:41.623218 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.622828    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 28 19:43:41.623748 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.622827    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 28 19:43:41.623853 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.623757    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nqr5x\""
May 28 19:43:41.623853 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.622987    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:41.623853 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.622925    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 28 19:43:41.624020 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.623021    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 28 19:43:41.624020 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.623012    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xh2xl\""
May 28 19:43:41.624020 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.623537    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 28 19:43:41.628719 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.628701    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 28 19:43:41.638227 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.638206    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b4944bb8f-8m9c7"]
May 28 19:43:41.638400 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.638380    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"
May 28 19:43:41.641268 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.641229    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-sr8d9\""
May 28 19:43:41.641510 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.641459    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:41.641776 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.641760    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 28 19:43:41.654082 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.654049    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-2t85s"]
May 28 19:43:41.680291 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.680240    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"]
May 28 19:43:41.680291 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.680283    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.680624 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.680357    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.683260 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683237    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 19:43:41.683391 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683288    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 19:43:41.683391 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683237    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kw6nh\""
May 28 19:43:41.683391 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683342    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 28 19:43:41.683391 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683242    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 19:43:41.683615 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683412    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vqg8p\""
May 28 19:43:41.683615 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683292    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 19:43:41.683713 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683615    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 19:43:41.683713 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.683677    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 28 19:43:41.689302 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.689186    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 28 19:43:41.690171 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.690149    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 19:43:41.695984 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.695958    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"]
May 28 19:43:41.695984 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.695992    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"]
May 28 19:43:41.696190 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.696090    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.699892 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.699866    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 28 19:43:41.700379 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.700360    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 19:43:41.700680 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.700659    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bbq54\""
May 28 19:43:41.700905 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.700884    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 19:43:41.701653 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.701631    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 28 19:43:41.708748 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.708722    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dc02322-296b-443f-bb2b-900dbdd4a4c8-serving-cert\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.708904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.708765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wzmv\" (UniqueName: \"kubernetes.io/projected/4e5384f0-d883-464f-8ac5-02cbdd363eb4-kube-api-access-4wzmv\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:41.708904 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.708826    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:41.709014 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.708919    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftnxv\" (UniqueName: \"kubernetes.io/projected/5dc02322-296b-443f-bb2b-900dbdd4a4c8-kube-api-access-ftnxv\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.709014 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.708994    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dc02322-296b-443f-bb2b-900dbdd4a4c8-trusted-ca\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.709114 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.709042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dc02322-296b-443f-bb2b-900dbdd4a4c8-config\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.726149 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.726110    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58c59b4dd8-8lv8f"]
May 28 19:43:41.726313 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.726291    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.729681 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.728969    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:41.729681 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.729031    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 28 19:43:41.729681 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.729251    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 28 19:43:41.729681 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.729292    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 28 19:43:41.729681 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.729523    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wsxql\""
May 28 19:43:41.741828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.741801    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"]
May 28 19:43:41.741975 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.741962    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:41.745548 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745242    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
May 28 19:43:41.745548 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745289    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 19:43:41.745548 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745454    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 19:43:41.745548 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745542    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
May 28 19:43:41.745835 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745648    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
May 28 19:43:41.745835 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745748    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-snbsh\""
May 28 19:43:41.745835 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.745759    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
May 28 19:43:41.760308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.760272    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-489wq"]
May 28 19:43:41.760709 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.760686    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:41.763543 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.763523    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 19:43:41.763652 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.763641    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 19:43:41.763758 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.763744    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
May 28 19:43:41.763962 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.763944    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 19:43:41.779972 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.779943    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"]
May 28 19:43:41.780128 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.780111    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-489wq"
May 28 19:43:41.783859 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.783830    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mqs6v\""
May 28 19:43:41.784017 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.783884    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 19:43:41.784112 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.784020    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 19:43:41.804052 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.804022    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lvcmf"]
May 28 19:43:41.804228 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.804173    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"
May 28 19:43:41.806640 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.806613    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 19:43:41.806798 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.806646    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 19:43:41.806798 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.806657    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-s92nh\""
May 28 19:43:41.809944 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.809914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.810083 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.809951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a5f910-b419-4fde-afac-6e0adc35d8a4-ca-trust-extracted\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.810083 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.809991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63e20a96-5b18-4200-99ee-4d2d35606b95-tmp\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.810083 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810035    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dc02322-296b-443f-bb2b-900dbdd4a4c8-serving-cert\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810081    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4wzmv\" (UniqueName: \"kubernetes.io/projected/4e5384f0-d883-464f-8ac5-02cbdd363eb4-kube-api-access-4wzmv\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810111    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810155    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda09b67-e01f-4af9-91a4-6aa913ab86d8-config\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810180    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/63e20a96-5b18-4200-99ee-4d2d35606b95-snapshots\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810230    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bda09b67-e01f-4af9-91a4-6aa913ab86d8-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810261    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ftnxv\" (UniqueName: \"kubernetes.io/projected/5dc02322-296b-443f-bb2b-900dbdd4a4c8-kube-api-access-ftnxv\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.810308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810313    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63e20a96-5b18-4200-99ee-4d2d35606b95-service-ca-bundle\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810334    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63e20a96-5b18-4200-99ee-4d2d35606b95-serving-cert\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810364    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg5jl\" (UniqueName: \"kubernetes.io/projected/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-kube-api-access-wg5jl\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810388    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-image-registry-private-configuration\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810416    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dc02322-296b-443f-bb2b-900dbdd4a4c8-config\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810438    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2kfp\" (UniqueName: \"kubernetes.io/projected/63e20a96-5b18-4200-99ee-4d2d35606b95-kube-api-access-c2kfp\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810506    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63e20a96-5b18-4200-99ee-4d2d35606b95-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810536    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-bound-sa-token\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810558    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.810674 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810589    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-installation-pull-secrets\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.810695    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810744    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-trusted-ca\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810793    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdmcv\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-kube-api-access-hdmcv\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.810880    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls podName:4e5384f0-d883-464f-8ac5-02cbdd363eb4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.31085689 +0000 UTC m=+35.598197796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls") pod "cluster-samples-operator-697587599-c8pj5" (UID: "4e5384f0-d883-464f-8ac5-02cbdd363eb4") : secret "samples-operator-tls" not found
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810934    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dc02322-296b-443f-bb2b-900dbdd4a4c8-trusted-ca\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.810985    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x4qt\" (UniqueName: \"kubernetes.io/projected/bda09b67-e01f-4af9-91a4-6aa913ab86d8-kube-api-access-7x4qt\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.811139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.811103    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-certificates\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.811511 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.811147    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vscck\" (UniqueName: \"kubernetes.io/projected/47aee4c8-e1ee-4f87-9e23-63f3146f9608-kube-api-access-vscck\") pod \"volume-data-source-validator-5cf8c89d64-mgxxg\" (UID: \"47aee4c8-e1ee-4f87-9e23-63f3146f9608\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"
May 28 19:43:41.811719 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.811696    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5dc02322-296b-443f-bb2b-900dbdd4a4c8-config\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.811958 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.811912    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5dc02322-296b-443f-bb2b-900dbdd4a4c8-trusted-ca\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.815176 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.815024    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5dc02322-296b-443f-bb2b-900dbdd4a4c8-serving-cert\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.824942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.824914    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftnxv\" (UniqueName: \"kubernetes.io/projected/5dc02322-296b-443f-bb2b-900dbdd4a4c8-kube-api-access-ftnxv\") pod \"console-operator-76dff48d59-9vlfb\" (UID: \"5dc02322-296b-443f-bb2b-900dbdd4a4c8\") " pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.825062 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.824914    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wzmv\" (UniqueName: \"kubernetes.io/projected/4e5384f0-d883-464f-8ac5-02cbdd363eb4-kube-api-access-4wzmv\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:41.825455 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.825433    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"]
May 28 19:43:41.825617 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.825598    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:41.828248 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.828152    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 19:43:41.828742 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.828712    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 19:43:41.828868 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.828179    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 19:43:41.828986 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.828972    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mmzwh\""
May 28 19:43:41.843449 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.843424    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"]
May 28 19:43:41.843622 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.843604    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:41.846316 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.846292    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
May 28 19:43:41.846494 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.846424    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
May 28 19:43:41.846494 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.846427    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
May 28 19:43:41.846776 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.846756    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
May 28 19:43:41.863703 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.863604    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"]
May 28 19:43:41.863845 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.863746    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:41.866449 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.866428    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kq9pt\""
May 28 19:43:41.866706 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.866486    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 19:43:41.885671 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885645    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"]
May 28 19:43:41.885801 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885777    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-9vlfb"]
May 28 19:43:41.885801 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885788    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:41.885801 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885802    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-2t85s"]
May 28 19:43:41.885956 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885812    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"]
May 28 19:43:41.885956 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885824    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b4944bb8f-8m9c7"]
May 28 19:43:41.885956 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885836    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"]
May 28 19:43:41.885956 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.885849    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-489wq"]
May 28 19:43:41.886120 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886039    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"]
May 28 19:43:41.886120 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886062    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"]
May 28 19:43:41.886120 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886076    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58c59b4dd8-8lv8f"]
May 28 19:43:41.886120 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886091    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvcmf"]
May 28 19:43:41.886120 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886105    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"]
May 28 19:43:41.886120 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886120    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"]
May 28 19:43:41.886349 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.886133    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"]
May 28 19:43:41.888484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.888439    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 28 19:43:41.888484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.888452    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 28 19:43:41.888484 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.888463    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 28 19:43:41.888734 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.888439    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-nk2d7\""
May 28 19:43:41.888734 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.888690    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 28 19:43:41.911621 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911595    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda09b67-e01f-4af9-91a4-6aa913ab86d8-config\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.911621 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911624    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/63e20a96-5b18-4200-99ee-4d2d35606b95-snapshots\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911646    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bda09b67-e01f-4af9-91a4-6aa913ab86d8-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911665    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911685    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63e20a96-5b18-4200-99ee-4d2d35606b95-service-ca-bundle\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911706    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63e20a96-5b18-4200-99ee-4d2d35606b95-serving-cert\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911737    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-default-certificate\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911764    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-stats-auth\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911802    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wg5jl\" (UniqueName: \"kubernetes.io/projected/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-kube-api-access-wg5jl\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.911834 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911828    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-image-registry-private-configuration\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911859    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c2kfp\" (UniqueName: \"kubernetes.io/projected/63e20a96-5b18-4200-99ee-4d2d35606b95-kube-api-access-c2kfp\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911890    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63e20a96-5b18-4200-99ee-4d2d35606b95-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911918    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-tmp\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911945    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-klusterlet-config\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.911972    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912005    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-bound-sa-token\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912031    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912057    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-installation-pull-secrets\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912109    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-trusted-ca\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912150    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hdmcv\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-kube-api-access-hdmcv\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912173    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/63e20a96-5b18-4200-99ee-4d2d35606b95-snapshots\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912177    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ecf972-8310-46b3-93b8-37284fcd1e04-tmp-dir\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912208    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bda09b67-e01f-4af9-91a4-6aa913ab86d8-config\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.912293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912656    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7x4qt\" (UniqueName: \"kubernetes.io/projected/bda09b67-e01f-4af9-91a4-6aa913ab86d8-kube-api-access-7x4qt\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912693    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-certificates\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912725    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7ns2\" (UniqueName: \"kubernetes.io/projected/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-kube-api-access-d7ns2\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912743    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63e20a96-5b18-4200-99ee-4d2d35606b95-service-ca-bundle\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912759    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vscck\" (UniqueName: \"kubernetes.io/projected/47aee4c8-e1ee-4f87-9e23-63f3146f9608-kube-api-access-vscck\") pod \"volume-data-source-validator-5cf8c89d64-mgxxg\" (UID: \"47aee4c8-e1ee-4f87-9e23-63f3146f9608\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912790    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlt46\" (UniqueName: \"kubernetes.io/projected/0c57594e-7784-4885-ab2f-577790157a3f-kube-api-access-wlt46\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9ecf972-8310-46b3-93b8-37284fcd1e04-config-volume\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912844    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmq9l\" (UniqueName: \"kubernetes.io/projected/d9ecf972-8310-46b3-93b8-37284fcd1e04-kube-api-access-jmq9l\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912879    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:41.912951 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.912923    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.913404 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.913001    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 19:43:41.913404 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.913067    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls podName:561fea3e-dfb5-42b2-b1d1-0581fec1d6eb nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.413048963 +0000 UTC m=+35.700389860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-mq7cf" (UID: "561fea3e-dfb5-42b2-b1d1-0581fec1d6eb") : secret "cluster-monitoring-operator-tls" not found
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.913700    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.913799    2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.913813    2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4944bb8f-8m9c7: secret "image-registry-tls" not found
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:41.913863    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls podName:f8a5f910-b419-4fde-afac-6e0adc35d8a4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.413845488 +0000 UTC m=+35.701186373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls") pod "image-registry-b4944bb8f-8m9c7" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4") : secret "image-registry-tls" not found
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.914364    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-certificates\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.914411    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a5f910-b419-4fde-afac-6e0adc35d8a4-ca-trust-extracted\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.914683 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.914500    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63e20a96-5b18-4200-99ee-4d2d35606b95-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.915126 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.914755    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63e20a96-5b18-4200-99ee-4d2d35606b95-tmp\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.915126 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.914805    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj4sg\" (UniqueName: \"kubernetes.io/projected/a09f5486-a17a-4554-a6d6-af3c724b747e-kube-api-access-jj4sg\") pod \"network-check-source-7f44b6dd87-7fmqd\" (UID: \"a09f5486-a17a-4554-a6d6-af3c724b747e\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"
May 28 19:43:41.915126 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.914913    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a5f910-b419-4fde-afac-6e0adc35d8a4-ca-trust-extracted\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.915126 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.915031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:41.915126 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.915053    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63e20a96-5b18-4200-99ee-4d2d35606b95-tmp\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.915126 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.915067    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4whm\" (UniqueName: \"kubernetes.io/projected/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-kube-api-access-v4whm\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:41.915869 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.915845    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-trusted-ca\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.916112 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.916089    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-image-registry-private-configuration\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.916219 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.916202    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-installation-pull-secrets\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.916366 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.916346    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bda09b67-e01f-4af9-91a4-6aa913ab86d8-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.918031 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.918008    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63e20a96-5b18-4200-99ee-4d2d35606b95-serving-cert\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.922566 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.922543    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdmcv\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-kube-api-access-hdmcv\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.925811 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.923126    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2kfp\" (UniqueName: \"kubernetes.io/projected/63e20a96-5b18-4200-99ee-4d2d35606b95-kube-api-access-c2kfp\") pod \"insights-operator-78bdfb888c-2t85s\" (UID: \"63e20a96-5b18-4200-99ee-4d2d35606b95\") " pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:41.925811 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.923317    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg5jl\" (UniqueName: \"kubernetes.io/projected/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-kube-api-access-wg5jl\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:41.925811 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.924087    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vscck\" (UniqueName: \"kubernetes.io/projected/47aee4c8-e1ee-4f87-9e23-63f3146f9608-kube-api-access-vscck\") pod \"volume-data-source-validator-5cf8c89d64-mgxxg\" (UID: \"47aee4c8-e1ee-4f87-9e23-63f3146f9608\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"
May 28 19:43:41.925811 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.924164    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x4qt\" (UniqueName: \"kubernetes.io/projected/bda09b67-e01f-4af9-91a4-6aa913ab86d8-kube-api-access-7x4qt\") pod \"kube-storage-version-migrator-operator-565644f49-r5f46\" (UID: \"bda09b67-e01f-4af9-91a4-6aa913ab86d8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:41.925811 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.924789    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-bound-sa-token\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:41.940032 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.939999    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:41.949879 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:41.949847    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"
May 28 19:43:42.001450 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.001398    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-2t85s"
May 28 19:43:42.015792 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015749    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-default-certificate\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.015965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015802    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-stats-auth\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.015965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015838    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/bb42dec1-4f99-4d22-a322-26acf79d0d47-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.015965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015876    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrjmm\" (UniqueName: \"kubernetes.io/projected/24426509-678b-4720-a347-531991889f77-kube-api-access-qrjmm\") pod \"managed-serviceaccount-addon-agent-8946c88cb-m2rtf\" (UID: \"24426509-678b-4720-a347-531991889f77\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.015965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24426509-678b-4720-a347-531991889f77-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8946c88cb-m2rtf\" (UID: \"24426509-678b-4720-a347-531991889f77\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.015965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015937    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2vh6\" (UniqueName: \"kubernetes.io/projected/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-kube-api-access-n2vh6\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.015965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015957    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-tmp\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015972    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-klusterlet-config\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.015989    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016043    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ecf972-8310-46b3-93b8-37284fcd1e04-tmp-dir\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016063    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-ca\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016101    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d7ns2\" (UniqueName: \"kubernetes.io/projected/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-kube-api-access-d7ns2\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-serving-cert\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016151    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wlt46\" (UniqueName: \"kubernetes.io/projected/0c57594e-7784-4885-ab2f-577790157a3f-kube-api-access-wlt46\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016166    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9ecf972-8310-46b3-93b8-37284fcd1e04-config-volume\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016182    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jmq9l\" (UniqueName: \"kubernetes.io/projected/d9ecf972-8310-46b3-93b8-37284fcd1e04-kube-api-access-jmq9l\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxdlw\" (UniqueName: \"kubernetes.io/projected/bb42dec1-4f99-4d22-a322-26acf79d0d47-kube-api-access-pxdlw\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.016271 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016238    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-config\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016279    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016306    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016346    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-tmp\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016365    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jj4sg\" (UniqueName: \"kubernetes.io/projected/a09f5486-a17a-4554-a6d6-af3c724b747e-kube-api-access-jj4sg\") pod \"network-check-source-7f44b6dd87-7fmqd\" (UID: \"a09f5486-a17a-4554-a6d6-af3c724b747e\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016377    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9ecf972-8310-46b3-93b8-37284fcd1e04-tmp-dir\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016446    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016509    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v4whm\" (UniqueName: \"kubernetes.io/projected/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-kube-api-access-v4whm\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016547    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-hub\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.016572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.016630    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.516611597 +0000 UTC m=+35.803952517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : configmap references non-existent config key: service-ca.crt
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.016767    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.016823    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls podName:d9ecf972-8310-46b3-93b8-37284fcd1e04 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.51680861 +0000 UTC m=+35.804149492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls") pod "dns-default-489wq" (UID: "d9ecf972-8310-46b3-93b8-37284fcd1e04") : secret "dns-default-metrics-tls" not found
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.016855    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.016891    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.516879562 +0000 UTC m=+35.804220448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : secret "router-metrics-certs-default" not found
May 28 19:43:42.017146 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.016971    2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 19:43:42.017651 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.017000    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert podName:0c57594e-7784-4885-ab2f-577790157a3f nodeName:}" failed. No retries permitted until 2026-05-28 19:43:42.516991261 +0000 UTC m=+35.804332143 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert") pod "ingress-canary-lvcmf" (UID: "0c57594e-7784-4885-ab2f-577790157a3f") : secret "canary-serving-cert" not found
May 28 19:43:42.017651 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.017370    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d9ecf972-8310-46b3-93b8-37284fcd1e04-config-volume\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.018705 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.018687    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-default-certificate\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.018867 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.018848    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-stats-auth\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.019780 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.019753    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-klusterlet-config\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.030408 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.030373    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmq9l\" (UniqueName: \"kubernetes.io/projected/d9ecf972-8310-46b3-93b8-37284fcd1e04-kube-api-access-jmq9l\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.030589 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.030530    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj4sg\" (UniqueName: \"kubernetes.io/projected/a09f5486-a17a-4554-a6d6-af3c724b747e-kube-api-access-jj4sg\") pod \"network-check-source-7f44b6dd87-7fmqd\" (UID: \"a09f5486-a17a-4554-a6d6-af3c724b747e\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"
May 28 19:43:42.030957 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.030757    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7ns2\" (UniqueName: \"kubernetes.io/projected/ed302eb8-1e9e-40d8-a5f1-b6962570e1d9-kube-api-access-d7ns2\") pod \"klusterlet-addon-workmgr-69c8cbd64f-sv8t8\" (UID: \"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.030957 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.030892    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlt46\" (UniqueName: \"kubernetes.io/projected/0c57594e-7784-4885-ab2f-577790157a3f-kube-api-access-wlt46\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:42.031230 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.031212    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4whm\" (UniqueName: \"kubernetes.io/projected/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-kube-api-access-v4whm\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.038891 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.038871    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"
May 28 19:43:42.071438 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.071401    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:42.114670 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.114587    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"
May 28 19:43:42.117744 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117720    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qrjmm\" (UniqueName: \"kubernetes.io/projected/24426509-678b-4720-a347-531991889f77-kube-api-access-qrjmm\") pod \"managed-serviceaccount-addon-agent-8946c88cb-m2rtf\" (UID: \"24426509-678b-4720-a347-531991889f77\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.117862 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117757    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24426509-678b-4720-a347-531991889f77-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8946c88cb-m2rtf\" (UID: \"24426509-678b-4720-a347-531991889f77\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.117862 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117779    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n2vh6\" (UniqueName: \"kubernetes.io/projected/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-kube-api-access-n2vh6\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.117971 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117873    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-ca\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.117971 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117902    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-serving-cert\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.117971 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117953    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxdlw\" (UniqueName: \"kubernetes.io/projected/bb42dec1-4f99-4d22-a322-26acf79d0d47-kube-api-access-pxdlw\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.118173 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.117977    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-config\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.118173 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.118001    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.118173 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.118080    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-hub\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.118173 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.118108    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.118173 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.118158    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/bb42dec1-4f99-4d22-a322-26acf79d0d47-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.118903 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.118880    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/bb42dec1-4f99-4d22-a322-26acf79d0d47-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.119270 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.119248    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-config\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.121227 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.121184    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/24426509-678b-4720-a347-531991889f77-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8946c88cb-m2rtf\" (UID: \"24426509-678b-4720-a347-531991889f77\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.121343 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.121227    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-ca\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.121343 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.121294    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-serving-cert\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.121649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.121622    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.122032 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.121997    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-hub\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.122390 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.122369    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/bb42dec1-4f99-4d22-a322-26acf79d0d47-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.126955 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.126934    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxdlw\" (UniqueName: \"kubernetes.io/projected/bb42dec1-4f99-4d22-a322-26acf79d0d47-kube-api-access-pxdlw\") pod \"cluster-proxy-proxy-agent-685496b79d-nmlrm\" (UID: \"bb42dec1-4f99-4d22-a322-26acf79d0d47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.127363 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.127343    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrjmm\" (UniqueName: \"kubernetes.io/projected/24426509-678b-4720-a347-531991889f77-kube-api-access-qrjmm\") pod \"managed-serviceaccount-addon-agent-8946c88cb-m2rtf\" (UID: \"24426509-678b-4720-a347-531991889f77\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.127432 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.127417    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2vh6\" (UniqueName: \"kubernetes.io/projected/168f7593-5ec2-4f21-9aa9-c4e4cebe52f5-kube-api-access-n2vh6\") pod \"service-ca-operator-57b5b68b8-kx6jt\" (UID: \"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.161590 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.161548    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:43:42.174400 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.174360    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"
May 28 19:43:42.217441 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.217408    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"
May 28 19:43:42.320444 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.320402    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:42.320662 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.320590    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 19:43:42.320726 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.320671    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls podName:4e5384f0-d883-464f-8ac5-02cbdd363eb4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.320650679 +0000 UTC m=+36.607991563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls") pod "cluster-samples-operator-697587599-c8pj5" (UID: "4e5384f0-d883-464f-8ac5-02cbdd363eb4") : secret "samples-operator-tls" not found
May 28 19:43:42.330367 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.329553    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:43:42.332721 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.332700    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-622pq\""
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.423199    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.423327    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.423543    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.423606    2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.423626    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls podName:561fea3e-dfb5-42b2-b1d1-0581fec1d6eb nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.423605319 +0000 UTC m=+36.710946221 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-mq7cf" (UID: "561fea3e-dfb5-42b2-b1d1-0581fec1d6eb") : secret "cluster-monitoring-operator-tls" not found
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.423630    2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4944bb8f-8m9c7: secret "image-registry-tls" not found
May 28 19:43:42.424169 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.423700    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls podName:f8a5f910-b419-4fde-afac-6e0adc35d8a4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.423682794 +0000 UTC m=+36.711023693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls") pod "image-registry-b4944bb8f-8m9c7" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4") : secret "image-registry-tls" not found
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.524207    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.524285    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.524322    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.524381    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.524561    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.524544335 +0000 UTC m=+36.811885217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : configmap references non-existent config key: service-ca.crt
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.525001    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.525040    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls podName:d9ecf972-8310-46b3-93b8-37284fcd1e04 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.525028502 +0000 UTC m=+36.812369386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls") pod "dns-default-489wq" (UID: "d9ecf972-8310-46b3-93b8-37284fcd1e04") : secret "dns-default-metrics-tls" not found
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.525084    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.525110    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.525101985 +0000 UTC m=+36.812442867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : secret "router-metrics-certs-default" not found
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.525149    2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 19:43:42.525203 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:42.525171    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert podName:0c57594e-7784-4885-ab2f-577790157a3f nodeName:}" failed. No retries permitted until 2026-05-28 19:43:43.52516443 +0000 UTC m=+36.812505314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert") pod "ingress-canary-lvcmf" (UID: "0c57594e-7784-4885-ab2f-577790157a3f") : secret "canary-serving-cert" not found
May 28 19:43:42.673001 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.672963    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-9vlfb"]
May 28 19:43:42.684718 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.684404    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-2t85s"]
May 28 19:43:42.691160 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.690553    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e20a96_5b18_4200_99ee_4d2d35606b95.slice/crio-2f845589f3ee39de2782b493bff86485c5be632f3aaeb185d0c73e32d0a06687 WatchSource:0}: Error finding container 2f845589f3ee39de2782b493bff86485c5be632f3aaeb185d0c73e32d0a06687: Status 404 returned error can't find the container with id 2f845589f3ee39de2782b493bff86485c5be632f3aaeb185d0c73e32d0a06687
May 28 19:43:42.693002 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.692541    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt"]
May 28 19:43:42.700311 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.700148    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46"]
May 28 19:43:42.701979 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.701167    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg"]
May 28 19:43:42.706429 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.706395    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47aee4c8_e1ee_4f87_9e23_63f3146f9608.slice/crio-38ce5281982f355a35927f658bf79726c7fc10cd61cd0ab8034e3dee81160d04 WatchSource:0}: Error finding container 38ce5281982f355a35927f658bf79726c7fc10cd61cd0ab8034e3dee81160d04: Status 404 returned error can't find the container with id 38ce5281982f355a35927f658bf79726c7fc10cd61cd0ab8034e3dee81160d04
May 28 19:43:42.712119 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.712096    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda09b67_e01f_4af9_91a4_6aa913ab86d8.slice/crio-397c190df6194ecbec4dc3357353e33565a9e72e485c55c3fb1d2cbf965048f3 WatchSource:0}: Error finding container 397c190df6194ecbec4dc3357353e33565a9e72e485c55c3fb1d2cbf965048f3: Status 404 returned error can't find the container with id 397c190df6194ecbec4dc3357353e33565a9e72e485c55c3fb1d2cbf965048f3
May 28 19:43:42.712786 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.712731    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"]
May 28 19:43:42.717195 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.717149    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf"]
May 28 19:43:42.717584 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.717547    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded302eb8_1e9e_40d8_a5f1_b6962570e1d9.slice/crio-e31d2aceef8bbf1db9a2d3fcc0a876a73b5a307c7ffa41901d8ef32c70dd1932 WatchSource:0}: Error finding container e31d2aceef8bbf1db9a2d3fcc0a876a73b5a307c7ffa41901d8ef32c70dd1932: Status 404 returned error can't find the container with id e31d2aceef8bbf1db9a2d3fcc0a876a73b5a307c7ffa41901d8ef32c70dd1932
May 28 19:43:42.721988 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.721946    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24426509_678b_4720_a347_531991889f77.slice/crio-8014058ed083497b84d34239a1acbfebdb6010de1f892103af7d1bc3b557a314 WatchSource:0}: Error finding container 8014058ed083497b84d34239a1acbfebdb6010de1f892103af7d1bc3b557a314: Status 404 returned error can't find the container with id 8014058ed083497b84d34239a1acbfebdb6010de1f892103af7d1bc3b557a314
May 28 19:43:42.729279 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.729240    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"]
May 28 19:43:42.731649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:42.731630    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd"]
May 28 19:43:42.740194 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.740168    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb42dec1_4f99_4d22_a322_26acf79d0d47.slice/crio-2f355ce5dfa8f8b8a848e29ee9fff8bb3ef9746d275da981889aa669580af23d WatchSource:0}: Error finding container 2f355ce5dfa8f8b8a848e29ee9fff8bb3ef9746d275da981889aa669580af23d: Status 404 returned error can't find the container with id 2f355ce5dfa8f8b8a848e29ee9fff8bb3ef9746d275da981889aa669580af23d
May 28 19:43:42.740927 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:42.740904    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda09f5486_a17a_4554_a6d6_af3c724b747e.slice/crio-6d6cf71e9fb33caaf788bec038e34475606e6625ca7b82eee7a5f5b97a48882b WatchSource:0}: Error finding container 6d6cf71e9fb33caaf788bec038e34475606e6625ca7b82eee7a5f5b97a48882b: Status 404 returned error can't find the container with id 6d6cf71e9fb33caaf788bec038e34475606e6625ca7b82eee7a5f5b97a48882b
May 28 19:43:43.330104 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.329839    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:43:43.330664 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.329897    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:43.332895 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.332870    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 19:43:43.333021 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.332913    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:43.333109 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.333087    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 19:43:43.333168 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.333159    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls podName:4e5384f0-d883-464f-8ac5-02cbdd363eb4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.333138297 +0000 UTC m=+38.620479180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls") pod "cluster-samples-operator-697587599-c8pj5" (UID: "4e5384f0-d883-464f-8ac5-02cbdd363eb4") : secret "samples-operator-tls" not found
May 28 19:43:43.333232 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.333188    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 19:43:43.334002 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.333977    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-msmgn\""
May 28 19:43:43.433893 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.433855    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:43.434047 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.433976    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:43.434120 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.434096    2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 19:43:43.434120 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.434117    2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4944bb8f-8m9c7: secret "image-registry-tls" not found
May 28 19:43:43.434210 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.434144    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 19:43:43.434210 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.434187    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls podName:f8a5f910-b419-4fde-afac-6e0adc35d8a4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.434161873 +0000 UTC m=+38.721502775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls") pod "image-registry-b4944bb8f-8m9c7" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4") : secret "image-registry-tls" not found
May 28 19:43:43.434210 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.434208    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls podName:561fea3e-dfb5-42b2-b1d1-0581fec1d6eb nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.43419802 +0000 UTC m=+38.721538908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-mq7cf" (UID: "561fea3e-dfb5-42b2-b1d1-0581fec1d6eb") : secret "cluster-monitoring-operator-tls" not found
May 28 19:43:43.527364 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.527326    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt" event={"ID":"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5","Type":"ContainerStarted","Data":"3796a8d1891ca7163aca430e573ee655a741c9ed2998e9d7342faa57537e81b7"}
May 28 19:43:43.533099 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.532092    2568 generic.go:358] "Generic (PLEG): container finished" podID="615454c2-0260-4ded-a724-210769c2ebfc" containerID="2746200b458edf89fddde1b5e2ae3b0c9de28de97f2d857898a036a6cb07bb0c" exitCode=0
May 28 19:43:43.533099 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.532163    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerDied","Data":"2746200b458edf89fddde1b5e2ae3b0c9de28de97f2d857898a036a6cb07bb0c"}
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.534529    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.534629    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.534680    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.534757    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.534986    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.535049    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.535029654 +0000 UTC m=+38.822370539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : secret "router-metrics-certs-default" not found
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.535436    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.535504    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls podName:d9ecf972-8310-46b3-93b8-37284fcd1e04 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.535488114 +0000 UTC m=+38.822829001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls") pod "dns-default-489wq" (UID: "d9ecf972-8310-46b3-93b8-37284fcd1e04") : secret "dns-default-metrics-tls" not found
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.535568    2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.535600    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert podName:0c57594e-7784-4885-ab2f-577790157a3f nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.535589572 +0000 UTC m=+38.822930457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert") pod "ingress-canary-lvcmf" (UID: "0c57594e-7784-4885-ab2f-577790157a3f") : secret "canary-serving-cert" not found
May 28 19:43:43.535812 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:43.535770    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:45.535757965 +0000 UTC m=+38.823098851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : configmap references non-existent config key: service-ca.crt
May 28 19:43:43.539944 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.539883    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46" event={"ID":"bda09b67-e01f-4af9-91a4-6aa913ab86d8","Type":"ContainerStarted","Data":"397c190df6194ecbec4dc3357353e33565a9e72e485c55c3fb1d2cbf965048f3"}
May 28 19:43:43.542240 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.542184    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" event={"ID":"5dc02322-296b-443f-bb2b-900dbdd4a4c8","Type":"ContainerStarted","Data":"eb2e49317bbf53c5f9d7bf21bd35c89ba4966c2c0a0b1edb75e776ddfd29cd24"}
May 28 19:43:43.544976 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.544916    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-2t85s" event={"ID":"63e20a96-5b18-4200-99ee-4d2d35606b95","Type":"ContainerStarted","Data":"2f845589f3ee39de2782b493bff86485c5be632f3aaeb185d0c73e32d0a06687"}
May 28 19:43:43.549251 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.549196    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8" event={"ID":"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9","Type":"ContainerStarted","Data":"e31d2aceef8bbf1db9a2d3fcc0a876a73b5a307c7ffa41901d8ef32c70dd1932"}
May 28 19:43:43.551554 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.551504    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf" event={"ID":"24426509-678b-4720-a347-531991889f77","Type":"ContainerStarted","Data":"8014058ed083497b84d34239a1acbfebdb6010de1f892103af7d1bc3b557a314"}
May 28 19:43:43.555828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.555760    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg" event={"ID":"47aee4c8-e1ee-4f87-9e23-63f3146f9608","Type":"ContainerStarted","Data":"38ce5281982f355a35927f658bf79726c7fc10cd61cd0ab8034e3dee81160d04"}
May 28 19:43:43.575351 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.569440    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" event={"ID":"bb42dec1-4f99-4d22-a322-26acf79d0d47","Type":"ContainerStarted","Data":"2f355ce5dfa8f8b8a848e29ee9fff8bb3ef9746d275da981889aa669580af23d"}
May 28 19:43:43.578159 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:43.578114    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd" event={"ID":"a09f5486-a17a-4554-a6d6-af3c724b747e","Type":"ContainerStarted","Data":"6d6cf71e9fb33caaf788bec038e34475606e6625ca7b82eee7a5f5b97a48882b"}
May 28 19:43:44.615175 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:44.615119    2568 generic.go:358] "Generic (PLEG): container finished" podID="615454c2-0260-4ded-a724-210769c2ebfc" containerID="68f11114f9e73485a7a3c502a29a2549f36f64097159afffb41dcafd35adf1d0" exitCode=0
May 28 19:43:44.616099 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:44.615204    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerDied","Data":"68f11114f9e73485a7a3c502a29a2549f36f64097159afffb41dcafd35adf1d0"}
May 28 19:43:45.362454 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.362369    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:45.364132 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.362572    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 19:43:45.364132 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.362679    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls podName:4e5384f0-d883-464f-8ac5-02cbdd363eb4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.362654801 +0000 UTC m=+42.649995686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls") pod "cluster-samples-operator-697587599-c8pj5" (UID: "4e5384f0-d883-464f-8ac5-02cbdd363eb4") : secret "samples-operator-tls" not found
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.463578    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.463730    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.463897    2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.463913    2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4944bb8f-8m9c7: secret "image-registry-tls" not found
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.463972    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls podName:f8a5f910-b419-4fde-afac-6e0adc35d8a4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.463953676 +0000 UTC m=+42.751294562 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls") pod "image-registry-b4944bb8f-8m9c7" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4") : secret "image-registry-tls" not found
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.464562    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 19:43:45.464740 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.464710    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls podName:561fea3e-dfb5-42b2-b1d1-0581fec1d6eb nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.464661535 +0000 UTC m=+42.752002420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-mq7cf" (UID: "561fea3e-dfb5-42b2-b1d1-0581fec1d6eb") : secret "cluster-monitoring-operator-tls" not found
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.565211    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.565307    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.565350    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.565419    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.565575    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.5655557 +0000 UTC m=+42.852896586 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : configmap references non-existent config key: service-ca.crt
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.565982    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.566029    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls podName:d9ecf972-8310-46b3-93b8-37284fcd1e04 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.566013682 +0000 UTC m=+42.853354569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls") pod "dns-default-489wq" (UID: "d9ecf972-8310-46b3-93b8-37284fcd1e04") : secret "dns-default-metrics-tls" not found
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.566088    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.566119    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.566107103 +0000 UTC m=+42.853447990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : secret "router-metrics-certs-default" not found
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.566167    2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 19:43:45.566233 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:45.566194    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert podName:0c57594e-7784-4885-ab2f-577790157a3f nodeName:}" failed. No retries permitted until 2026-05-28 19:43:49.56618468 +0000 UTC m=+42.853525566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert") pod "ingress-canary-lvcmf" (UID: "0c57594e-7784-4885-ab2f-577790157a3f") : secret "canary-serving-cert" not found
May 28 19:43:45.667461 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.666391    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" event={"ID":"615454c2-0260-4ded-a724-210769c2ebfc","Type":"ContainerStarted","Data":"476b9fabe4b75edbeb9438f45b3f0df27a497c1503f1427781484137305a89ba"}
May 28 19:43:45.692667 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:45.690933    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j7t6m" podStartSLOduration=6.09552535 podStartE2EDuration="38.690899265s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:43:10.030375793 +0000 UTC m=+3.317716675" lastFinishedPulling="2026-05-28 19:43:42.625749694 +0000 UTC m=+35.913090590" observedRunningTime="2026-05-28 19:43:45.689729787 +0000 UTC m=+38.977070727" watchObservedRunningTime="2026-05-28 19:43:45.690899265 +0000 UTC m=+38.978240172"
May 28 19:43:48.092060 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:48.092019    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:48.096782 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:48.096750    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/11c4e6c3-c807-4c58-bf0a-551c264cb561-original-pull-secret\") pod \"global-pull-secret-syncer-px2n9\" (UID: \"11c4e6c3-c807-4c58-bf0a-551c264cb561\") " pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:48.150040 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:48.149996    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-px2n9"
May 28 19:43:49.404438 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.404399    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:49.405038 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.404588    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 19:43:49.405038 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.404653    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls podName:4e5384f0-d883-464f-8ac5-02cbdd363eb4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.404633299 +0000 UTC m=+50.691974183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls") pod "cluster-samples-operator-697587599-c8pj5" (UID: "4e5384f0-d883-464f-8ac5-02cbdd363eb4") : secret "samples-operator-tls" not found
May 28 19:43:49.505006 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.504960    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:49.505193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.505073    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:49.505193 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.505122    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 19:43:49.505309 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.505201    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls podName:561fea3e-dfb5-42b2-b1d1-0581fec1d6eb nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.505180206 +0000 UTC m=+50.792521104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-mq7cf" (UID: "561fea3e-dfb5-42b2-b1d1-0581fec1d6eb") : secret "cluster-monitoring-operator-tls" not found
May 28 19:43:49.505309 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.505204    2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 19:43:49.505309 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.505217    2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4944bb8f-8m9c7: secret "image-registry-tls" not found
May 28 19:43:49.505309 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.505252    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls podName:f8a5f910-b419-4fde-afac-6e0adc35d8a4 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.505240896 +0000 UTC m=+50.792581777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls") pod "image-registry-b4944bb8f-8m9c7" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4") : secret "image-registry-tls" not found
May 28 19:43:49.606384 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.606343    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:49.606586 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.606449    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:49.606586 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.606524    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:49.606586 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606538    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 19:43:49.606586 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606563    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 19:43:49.606779 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:49.606601    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:49.606779 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606620    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls podName:d9ecf972-8310-46b3-93b8-37284fcd1e04 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.606599774 +0000 UTC m=+50.893940679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls") pod "dns-default-489wq" (UID: "d9ecf972-8310-46b3-93b8-37284fcd1e04") : secret "dns-default-metrics-tls" not found
May 28 19:43:49.606779 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606657    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.606647625 +0000 UTC m=+50.893988507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : secret "router-metrics-certs-default" not found
May 28 19:43:49.606779 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606667    2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 19:43:49.606779 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606702    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.606685152 +0000 UTC m=+50.894026041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : configmap references non-existent config key: service-ca.crt
May 28 19:43:49.606779 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:49.606724    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert podName:0c57594e-7784-4885-ab2f-577790157a3f nodeName:}" failed. No retries permitted until 2026-05-28 19:43:57.606714478 +0000 UTC m=+50.894055366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert") pod "ingress-canary-lvcmf" (UID: "0c57594e-7784-4885-ab2f-577790157a3f") : secret "canary-serving-cert" not found
May 28 19:43:56.629291 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.629263    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-px2n9"]
May 28 19:43:56.693935 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.693878    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8" event={"ID":"ed302eb8-1e9e-40d8-a5f1-b6962570e1d9","Type":"ContainerStarted","Data":"56d193f6294cde1680921b24b3539443c6498d693b0313b8631b5c976aedf24e"}
May 28 19:43:56.695540 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.694712    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:56.696500 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.696124    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8"
May 28 19:43:56.696500 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.696403    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg" event={"ID":"47aee4c8-e1ee-4f87-9e23-63f3146f9608","Type":"ContainerStarted","Data":"7fa0562a2659a84134d1b7c1f5ea305cd3c96d5927fa201a27973c56a20e14d5"}
May 28 19:43:56.700493 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.700435    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd" event={"ID":"a09f5486-a17a-4554-a6d6-af3c724b747e","Type":"ContainerStarted","Data":"e4ec47b29e85195ed25071f2608f3460a276c80c4fdbd138f26031c1e2fe2187"}
May 28 19:43:56.702343 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.702318    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" event={"ID":"5dc02322-296b-443f-bb2b-900dbdd4a4c8","Type":"ContainerStarted","Data":"be93715b7e3113b5fda18a45f1daa366198c26e02b45f82167cfe196f1915cee"}
May 28 19:43:56.703535 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.703502    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:43:56.704384 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.704343    2568 patch_prober.go:28] interesting pod/console-operator-76dff48d59-9vlfb container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.7:8443/readyz\": dial tcp 10.132.0.7:8443: connect: connection refused" start-of-body=
May 28 19:43:56.704461 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.704402    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" podUID="5dc02322-296b-443f-bb2b-900dbdd4a4c8" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.7:8443/readyz\": dial tcp 10.132.0.7:8443: connect: connection refused"
May 28 19:43:56.717090 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.716568    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69c8cbd64f-sv8t8" podStartSLOduration=19.943365532 podStartE2EDuration="33.716552955s" podCreationTimestamp="2026-05-28 19:43:23 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.721252441 +0000 UTC m=+36.008593324" lastFinishedPulling="2026-05-28 19:43:56.494439864 +0000 UTC m=+49.781780747" observedRunningTime="2026-05-28 19:43:56.715748318 +0000 UTC m=+50.003089223" watchObservedRunningTime="2026-05-28 19:43:56.716552955 +0000 UTC m=+50.003893858"
May 28 19:43:56.744729 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.744680    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" podStartSLOduration=18.987722926 podStartE2EDuration="32.744666581s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.687235605 +0000 UTC m=+35.974576494" lastFinishedPulling="2026-05-28 19:43:56.444179262 +0000 UTC m=+49.731520149" observedRunningTime="2026-05-28 19:43:56.74376657 +0000 UTC m=+50.031107475" watchObservedRunningTime="2026-05-28 19:43:56.744666581 +0000 UTC m=+50.032007484"
May 28 19:43:56.790544 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:43:56.790507    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11c4e6c3_c807_4c58_bf0a_551c264cb561.slice/crio-32d21fc036148c464d7b7373aab9727b76e05a8ab52015fdd7c33ddf40b2829e WatchSource:0}: Error finding container 32d21fc036148c464d7b7373aab9727b76e05a8ab52015fdd7c33ddf40b2829e: Status 404 returned error can't find the container with id 32d21fc036148c464d7b7373aab9727b76e05a8ab52015fdd7c33ddf40b2829e
May 28 19:43:56.815168 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.815066    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-mgxxg" podStartSLOduration=19.079913769 podStartE2EDuration="32.815050372s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.709046626 +0000 UTC m=+35.996387508" lastFinishedPulling="2026-05-28 19:43:56.444183227 +0000 UTC m=+49.731524111" observedRunningTime="2026-05-28 19:43:56.814141465 +0000 UTC m=+50.101482372" watchObservedRunningTime="2026-05-28 19:43:56.815050372 +0000 UTC m=+50.102391278"
May 28 19:43:56.815527 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:56.815492    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-7fmqd" podStartSLOduration=18.064124287 podStartE2EDuration="31.815482175s" podCreationTimestamp="2026-05-28 19:43:25 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.742699627 +0000 UTC m=+36.030040512" lastFinishedPulling="2026-05-28 19:43:56.494057503 +0000 UTC m=+49.781398400" observedRunningTime="2026-05-28 19:43:56.793648736 +0000 UTC m=+50.080989641" watchObservedRunningTime="2026-05-28 19:43:56.815482175 +0000 UTC m=+50.102823076"
May 28 19:43:57.482491 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.482424    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:43:57.482724 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.482614    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 28 19:43:57.482724 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.482700    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls podName:4e5384f0-d883-464f-8ac5-02cbdd363eb4 nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.482678226 +0000 UTC m=+66.770019113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls") pod "cluster-samples-operator-697587599-c8pj5" (UID: "4e5384f0-d883-464f-8ac5-02cbdd363eb4") : secret "samples-operator-tls" not found
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.582948    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.583069    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.583097    2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.583115    2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b4944bb8f-8m9c7: secret "image-registry-tls" not found
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.583174    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls podName:f8a5f910-b419-4fde-afac-6e0adc35d8a4 nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.583156057 +0000 UTC m=+66.870496942 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls") pod "image-registry-b4944bb8f-8m9c7" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4") : secret "image-registry-tls" not found
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.583228    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 28 19:43:57.583293 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.583266    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls podName:561fea3e-dfb5-42b2-b1d1-0581fec1d6eb nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.583254398 +0000 UTC m=+66.870595282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-mq7cf" (UID: "561fea3e-dfb5-42b2-b1d1-0581fec1d6eb") : secret "cluster-monitoring-operator-tls" not found
May 28 19:43:57.683814 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.683737    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.683839    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.683883    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.683992    2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.683998    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.684052    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert podName:0c57594e-7784-4885-ab2f-577790157a3f nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.684032492 +0000 UTC m=+66.971373389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert") pod "ingress-canary-lvcmf" (UID: "0c57594e-7784-4885-ab2f-577790157a3f") : secret "canary-serving-cert" not found
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.684068    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls podName:d9ecf972-8310-46b3-93b8-37284fcd1e04 nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.684060381 +0000 UTC m=+66.971401267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls") pod "dns-default-489wq" (UID: "d9ecf972-8310-46b3-93b8-37284fcd1e04") : secret "dns-default-metrics-tls" not found
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.684104    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.684126    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.684142    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.684125496 +0000 UTC m=+66.971466385 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : secret "router-metrics-certs-default" not found
May 28 19:43:57.684262 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:57.684194    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle podName:e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54 nodeName:}" failed. No retries permitted until 2026-05-28 19:44:13.684184056 +0000 UTC m=+66.971524942 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle") pod "router-default-58c59b4dd8-8lv8f" (UID: "e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54") : configmap references non-existent config key: service-ca.crt
May 28 19:43:57.708568 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.707748    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/0.log"
May 28 19:43:57.708568 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.707786    2568 generic.go:358] "Generic (PLEG): container finished" podID="5dc02322-296b-443f-bb2b-900dbdd4a4c8" containerID="be93715b7e3113b5fda18a45f1daa366198c26e02b45f82167cfe196f1915cee" exitCode=255
May 28 19:43:57.708568 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.707834    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" event={"ID":"5dc02322-296b-443f-bb2b-900dbdd4a4c8","Type":"ContainerDied","Data":"be93715b7e3113b5fda18a45f1daa366198c26e02b45f82167cfe196f1915cee"}
May 28 19:43:57.708568 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.708215    2568 scope.go:117] "RemoveContainer" containerID="be93715b7e3113b5fda18a45f1daa366198c26e02b45f82167cfe196f1915cee"
May 28 19:43:57.711513 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.710523    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-2t85s" event={"ID":"63e20a96-5b18-4200-99ee-4d2d35606b95","Type":"ContainerStarted","Data":"a4b197dcfb4684aae81cbd63e09898a9754917441de7e021d8da9113c6c6bcbe"}
May 28 19:43:57.715073 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.714617    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf" event={"ID":"24426509-678b-4720-a347-531991889f77","Type":"ContainerStarted","Data":"6aa19683a86f5c5675b346307ad0f58c971b0a740b89ba1614a840d071e37df5"}
May 28 19:43:57.717749 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.717698    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" event={"ID":"bb42dec1-4f99-4d22-a322-26acf79d0d47","Type":"ContainerStarted","Data":"03eed69784b1a96467e6f20beffe9447e4af6f969c15c37a2f21133e064ad945"}
May 28 19:43:57.722743 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.722718    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-px2n9" event={"ID":"11c4e6c3-c807-4c58-bf0a-551c264cb561","Type":"ContainerStarted","Data":"32d21fc036148c464d7b7373aab9727b76e05a8ab52015fdd7c33ddf40b2829e"}
May 28 19:43:57.728880 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.728305    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt" event={"ID":"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5","Type":"ContainerStarted","Data":"54e01316997f205f143eb09cdc83978bbf82441142543f99fc2ba17b6ecfcdc6"}
May 28 19:43:57.733984 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.733784    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46" event={"ID":"bda09b67-e01f-4af9-91a4-6aa913ab86d8","Type":"ContainerStarted","Data":"a1e8cce90d4642ba45594ba812f435ee53ef149ebc9e34f5fe7b109584e63549"}
May 28 19:43:57.741197 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.740575    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8946c88cb-m2rtf" podStartSLOduration=20.96164418 podStartE2EDuration="34.740557178s" podCreationTimestamp="2026-05-28 19:43:23 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.724291976 +0000 UTC m=+36.011632859" lastFinishedPulling="2026-05-28 19:43:56.503204967 +0000 UTC m=+49.790545857" observedRunningTime="2026-05-28 19:43:57.738951999 +0000 UTC m=+51.026292926" watchObservedRunningTime="2026-05-28 19:43:57.740557178 +0000 UTC m=+51.027898082"
May 28 19:43:57.759005 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.757931    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-2t85s" podStartSLOduration=19.955979901 podStartE2EDuration="33.757914937s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.695034171 +0000 UTC m=+35.982375056" lastFinishedPulling="2026-05-28 19:43:56.496969203 +0000 UTC m=+49.784310092" observedRunningTime="2026-05-28 19:43:57.757594885 +0000 UTC m=+51.044935790" watchObservedRunningTime="2026-05-28 19:43:57.757914937 +0000 UTC m=+51.045255842"
May 28 19:43:57.784856 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.778725    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46" podStartSLOduration=19.998162736 podStartE2EDuration="33.778705118s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.713906984 +0000 UTC m=+36.001247871" lastFinishedPulling="2026-05-28 19:43:56.494449357 +0000 UTC m=+49.781790253" observedRunningTime="2026-05-28 19:43:57.778134425 +0000 UTC m=+51.065475330" watchObservedRunningTime="2026-05-28 19:43:57.778705118 +0000 UTC m=+51.066046023"
May 28 19:43:57.797134 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:57.796444    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt" podStartSLOduration=20.004370135 podStartE2EDuration="33.796426615s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.70233647 +0000 UTC m=+35.989677354" lastFinishedPulling="2026-05-28 19:43:56.494392952 +0000 UTC m=+49.781733834" observedRunningTime="2026-05-28 19:43:57.795069276 +0000 UTC m=+51.082410181" watchObservedRunningTime="2026-05-28 19:43:57.796426615 +0000 UTC m=+51.083767520"
May 28 19:43:58.738389 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:58.738358    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:43:58.740768 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:58.739102    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/0.log"
May 28 19:43:58.740768 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:58.739148    2568 generic.go:358] "Generic (PLEG): container finished" podID="5dc02322-296b-443f-bb2b-900dbdd4a4c8" containerID="f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83" exitCode=255
May 28 19:43:58.740768 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:58.740164    2568 scope.go:117] "RemoveContainer" containerID="f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83"
May 28 19:43:58.740768 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:58.740334    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-9vlfb_openshift-console-operator(5dc02322-296b-443f-bb2b-900dbdd4a4c8)\"" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" podUID="5dc02322-296b-443f-bb2b-900dbdd4a4c8"
May 28 19:43:58.740768 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:58.740556    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" event={"ID":"5dc02322-296b-443f-bb2b-900dbdd4a4c8","Type":"ContainerDied","Data":"f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83"}
May 28 19:43:58.740768 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:58.740585    2568 scope.go:117] "RemoveContainer" containerID="be93715b7e3113b5fda18a45f1daa366198c26e02b45f82167cfe196f1915cee"
May 28 19:43:59.744579 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:59.744554    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:43:59.744973 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:59.744927    2568 scope.go:117] "RemoveContainer" containerID="f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83"
May 28 19:43:59.745142 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:43:59.745117    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-9vlfb_openshift-console-operator(5dc02322-296b-443f-bb2b-900dbdd4a4c8)\"" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" podUID="5dc02322-296b-443f-bb2b-900dbdd4a4c8"
May 28 19:43:59.747581 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:43:59.747557    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" event={"ID":"bb42dec1-4f99-4d22-a322-26acf79d0d47","Type":"ContainerStarted","Data":"01fdbc8484f90b1ec93f012861f37eda8a9faf4fad77c6faf2ae5104e76d6c8e"}
May 28 19:44:00.753733 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:00.753688    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" event={"ID":"bb42dec1-4f99-4d22-a322-26acf79d0d47","Type":"ContainerStarted","Data":"317a979b7e646851fa4b8ffd003f67dcb982807317aedce55accaa15e4a63bd5"}
May 28 19:44:00.776056 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:00.776000    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" podStartSLOduration=20.905887672 podStartE2EDuration="37.77597962s" podCreationTimestamp="2026-05-28 19:43:23 +0000 UTC" firstStartedPulling="2026-05-28 19:43:42.742414901 +0000 UTC m=+36.029755783" lastFinishedPulling="2026-05-28 19:43:59.612506846 +0000 UTC m=+52.899847731" observedRunningTime="2026-05-28 19:44:00.775051651 +0000 UTC m=+54.062392558" watchObservedRunningTime="2026-05-28 19:44:00.77597962 +0000 UTC m=+54.063320523"
May 28 19:44:01.106095 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:01.106067    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rflwq_054a4749-b39d-4f52-b4cc-cc15a9a93bf8/dns-node-resolver/0.log"
May 28 19:44:01.758316 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:01.758271    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-px2n9" event={"ID":"11c4e6c3-c807-4c58-bf0a-551c264cb561","Type":"ContainerStarted","Data":"5b6262575dfb5775d821500b079112087955d04fc688cd57bd9b9634210b172e"}
May 28 19:44:01.776266 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:01.776205    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-px2n9" podStartSLOduration=41.007835313 podStartE2EDuration="45.776187449s" podCreationTimestamp="2026-05-28 19:43:16 +0000 UTC" firstStartedPulling="2026-05-28 19:43:56.793153083 +0000 UTC m=+50.080493972" lastFinishedPulling="2026-05-28 19:44:01.561505216 +0000 UTC m=+54.848846108" observedRunningTime="2026-05-28 19:44:01.775374406 +0000 UTC m=+55.062715311" watchObservedRunningTime="2026-05-28 19:44:01.776187449 +0000 UTC m=+55.063528353"
May 28 19:44:01.941012 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:01.940928    2568 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:44:01.941308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:01.941295    2568 scope.go:117] "RemoveContainer" containerID="f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83"
May 28 19:44:01.941511 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:44:01.941465    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-9vlfb_openshift-console-operator(5dc02322-296b-443f-bb2b-900dbdd4a4c8)\"" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" podUID="5dc02322-296b-443f-bb2b-900dbdd4a4c8"
May 28 19:44:02.507103 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:02.507073    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hl88v_bd7826c6-6b85-45d0-9a3a-c1b4fca63208/node-ca/0.log"
May 28 19:44:03.707863 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:03.707830    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-r5f46_bda09b67-e01f-4af9-91a4-6aa913ab86d8/kube-storage-version-migrator-operator/0.log"
May 28 19:44:06.703717 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:06.703685    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:44:06.704086 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:06.704057    2568 scope.go:117] "RemoveContainer" containerID="f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83"
May 28 19:44:06.704242 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:44:06.704225    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-9vlfb_openshift-console-operator(5dc02322-296b-443f-bb2b-900dbdd4a4c8)\"" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" podUID="5dc02322-296b-443f-bb2b-900dbdd4a4c8"
May 28 19:44:07.528823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:07.528794    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxvsp"
May 28 19:44:13.132305 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.132255    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:44:13.132890 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.132379    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:44:13.135162 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.135139    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db9kx\" (UniqueName: \"kubernetes.io/projected/e9e44915-c783-4fb0-84e0-1b0c0d67dfd2-kube-api-access-db9kx\") pod \"network-check-target-r9l4n\" (UID: \"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2\") " pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:44:13.135276 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.135166    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 19:44:13.145395 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.145364    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0f40971c-dd1a-4403-a51b-bdd13d1fb3cc-metrics-certs\") pod \"network-metrics-daemon-wxzjd\" (UID: \"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc\") " pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:44:13.242754 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.242717    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-622pq\""
May 28 19:44:13.250731 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.250706    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:44:13.345200 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.345172    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-msmgn\""
May 28 19:44:13.352541 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.352494    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-wxzjd"
May 28 19:44:13.381161 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.381127    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r9l4n"]
May 28 19:44:13.384603 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:13.384574    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e44915_c783_4fb0_84e0_1b0c0d67dfd2.slice/crio-b007b8dfe83c5945cb30a02345b9766a3ea8a9d5e7313749a931e23dc1064fc6 WatchSource:0}: Error finding container b007b8dfe83c5945cb30a02345b9766a3ea8a9d5e7313749a931e23dc1064fc6: Status 404 returned error can't find the container with id b007b8dfe83c5945cb30a02345b9766a3ea8a9d5e7313749a931e23dc1064fc6
May 28 19:44:13.483307 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.483277    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-wxzjd"]
May 28 19:44:13.495606 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:13.495578    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f40971c_dd1a_4403_a51b_bdd13d1fb3cc.slice/crio-c0525fcc13ba76981df9f7467ac87377675423cf0cbd3c1b95fc29edd5cd245a WatchSource:0}: Error finding container c0525fcc13ba76981df9f7467ac87377675423cf0cbd3c1b95fc29edd5cd245a: Status 404 returned error can't find the container with id c0525fcc13ba76981df9f7467ac87377675423cf0cbd3c1b95fc29edd5cd245a
May 28 19:44:13.536525 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.536461    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:44:13.538978 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.538958    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4e5384f0-d883-464f-8ac5-02cbdd363eb4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-c8pj5\" (UID: \"4e5384f0-d883-464f-8ac5-02cbdd363eb4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:44:13.637614 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.637521    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:44:13.637614 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.637606    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:44:13.640280 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.640249    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"image-registry-b4944bb8f-8m9c7\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") " pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:44:13.640406 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.640352    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/561fea3e-dfb5-42b2-b1d1-0581fec1d6eb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-mq7cf\" (UID: \"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:44:13.736410 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.736376    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xh2xl\""
May 28 19:44:13.738714 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.738695    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:13.738788 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.738735    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:44:13.738825 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.738781    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:13.738857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.738837    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:44:13.739526 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.739434    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-service-ca-bundle\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:13.741088 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.741069    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d9ecf972-8310-46b3-93b8-37284fcd1e04-metrics-tls\") pod \"dns-default-489wq\" (UID: \"d9ecf972-8310-46b3-93b8-37284fcd1e04\") " pod="openshift-dns/dns-default-489wq"
May 28 19:44:13.741183 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.741161    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c57594e-7784-4885-ab2f-577790157a3f-cert\") pod \"ingress-canary-lvcmf\" (UID: \"0c57594e-7784-4885-ab2f-577790157a3f\") " pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:44:13.741234 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.741218    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54-metrics-certs\") pod \"router-default-58c59b4dd8-8lv8f\" (UID: \"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54\") " pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:13.745074 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.745047    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"
May 28 19:44:13.788348 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.788304    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxzjd" event={"ID":"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc","Type":"ContainerStarted","Data":"c0525fcc13ba76981df9f7467ac87377675423cf0cbd3c1b95fc29edd5cd245a"}
May 28 19:44:13.789991 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.789954    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r9l4n" event={"ID":"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2","Type":"ContainerStarted","Data":"f6d76aac594c336c8fbf26e5aa6784b4056a96bfa627ffe42371a17d7565da59"}
May 28 19:44:13.790117 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.789999    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r9l4n" event={"ID":"e9e44915-c783-4fb0-84e0-1b0c0d67dfd2","Type":"ContainerStarted","Data":"b007b8dfe83c5945cb30a02345b9766a3ea8a9d5e7313749a931e23dc1064fc6"}
May 28 19:44:13.790811 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.790787    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:44:13.795754 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.795729    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kw6nh\""
May 28 19:44:13.804002 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.803973    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:44:13.808576 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.808535    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r9l4n" podStartSLOduration=66.808521807 podStartE2EDuration="1m6.808521807s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 19:44:13.807428786 +0000 UTC m=+67.094769690" watchObservedRunningTime="2026-05-28 19:44:13.808521807 +0000 UTC m=+67.095862713"
May 28 19:44:13.811546 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.811523    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bbq54\""
May 28 19:44:13.819787 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.819763    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"
May 28 19:44:13.855900 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.855837    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-snbsh\""
May 28 19:44:13.864540 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.863708    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:13.884984 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.884707    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5"]
May 28 19:44:13.895290 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.895227    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mqs6v\""
May 28 19:44:13.902571 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.902495    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-489wq"
May 28 19:44:13.947511 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.946528    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mmzwh\""
May 28 19:44:13.955972 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.954510    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvcmf"
May 28 19:44:13.968001 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.967963    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b4944bb8f-8m9c7"]
May 28 19:44:13.987856 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:13.985127    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf"]
May 28 19:44:13.999282 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:13.999230    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod561fea3e_dfb5_42b2_b1d1_0581fec1d6eb.slice/crio-eb22aa134e7c226d26f7e25d3997ed3a29b49335ba8ae9f4864efc6dd462acb8 WatchSource:0}: Error finding container eb22aa134e7c226d26f7e25d3997ed3a29b49335ba8ae9f4864efc6dd462acb8: Status 404 returned error can't find the container with id eb22aa134e7c226d26f7e25d3997ed3a29b49335ba8ae9f4864efc6dd462acb8
May 28 19:44:14.062823 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.062637    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58c59b4dd8-8lv8f"]
May 28 19:44:14.065459 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:14.065217    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2acbbe9_dff4_4b15_be5c_7eff8e9fdd54.slice/crio-c1dd0e8011bc114d5bd203c41100501db77a816316b08e8db45c0ebb658a81e3 WatchSource:0}: Error finding container c1dd0e8011bc114d5bd203c41100501db77a816316b08e8db45c0ebb658a81e3: Status 404 returned error can't find the container with id c1dd0e8011bc114d5bd203c41100501db77a816316b08e8db45c0ebb658a81e3
May 28 19:44:14.087297 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.087266    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-489wq"]
May 28 19:44:14.092493 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:14.092430    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ecf972_8310_46b3_93b8_37284fcd1e04.slice/crio-37a0c06fb34fca2f4d78ca2cd333ad3acb1459572f3bf1cc9f816db46055c84f WatchSource:0}: Error finding container 37a0c06fb34fca2f4d78ca2cd333ad3acb1459572f3bf1cc9f816db46055c84f: Status 404 returned error can't find the container with id 37a0c06fb34fca2f4d78ca2cd333ad3acb1459572f3bf1cc9f816db46055c84f
May 28 19:44:14.127589 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.127558    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvcmf"]
May 28 19:44:14.131549 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:14.131486    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c57594e_7784_4885_ab2f_577790157a3f.slice/crio-a20086d2c3206051ea1663cd576bea8a4abaa0fae9c62fd38e9b8bfad9f9bcd4 WatchSource:0}: Error finding container a20086d2c3206051ea1663cd576bea8a4abaa0fae9c62fd38e9b8bfad9f9bcd4: Status 404 returned error can't find the container with id a20086d2c3206051ea1663cd576bea8a4abaa0fae9c62fd38e9b8bfad9f9bcd4
May 28 19:44:14.796267 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.796215    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvcmf" event={"ID":"0c57594e-7784-4885-ab2f-577790157a3f","Type":"ContainerStarted","Data":"a20086d2c3206051ea1663cd576bea8a4abaa0fae9c62fd38e9b8bfad9f9bcd4"}
May 28 19:44:14.798363 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.798313    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-489wq" event={"ID":"d9ecf972-8310-46b3-93b8-37284fcd1e04","Type":"ContainerStarted","Data":"37a0c06fb34fca2f4d78ca2cd333ad3acb1459572f3bf1cc9f816db46055c84f"}
May 28 19:44:14.800177 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.800142    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f" event={"ID":"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54","Type":"ContainerStarted","Data":"e23131ad63208d26d8dc670619bb88fd05cb7365e5489920f97822040037f942"}
May 28 19:44:14.800310 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.800187    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f" event={"ID":"e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54","Type":"ContainerStarted","Data":"c1dd0e8011bc114d5bd203c41100501db77a816316b08e8db45c0ebb658a81e3"}
May 28 19:44:14.802518 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.802425    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf" event={"ID":"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb","Type":"ContainerStarted","Data":"eb22aa134e7c226d26f7e25d3997ed3a29b49335ba8ae9f4864efc6dd462acb8"}
May 28 19:44:14.804967 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.804882    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" event={"ID":"f8a5f910-b419-4fde-afac-6e0adc35d8a4","Type":"ContainerStarted","Data":"fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3"}
May 28 19:44:14.804967 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.804915    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" event={"ID":"f8a5f910-b419-4fde-afac-6e0adc35d8a4","Type":"ContainerStarted","Data":"f4e330bbf6511605772290b1340826045b7a1e9fe1f92dcad615ac112a79fa5f"}
May 28 19:44:14.805257 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.805214    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:44:14.806999 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.806967    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5" event={"ID":"4e5384f0-d883-464f-8ac5-02cbdd363eb4","Type":"ContainerStarted","Data":"f0bdc6aba007516f0ace0ed641e34a4f5a0eae6fda36bf22b3774c9a22977347"}
May 28 19:44:14.822682 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.822004    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f" podStartSLOduration=50.821983504 podStartE2EDuration="50.821983504s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 19:44:14.820154821 +0000 UTC m=+68.107495727" watchObservedRunningTime="2026-05-28 19:44:14.821983504 +0000 UTC m=+68.109324406"
May 28 19:44:14.839023 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.838940    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" podStartSLOduration=66.838922326 podStartE2EDuration="1m6.838922326s" podCreationTimestamp="2026-05-28 19:43:08 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 19:44:14.838439402 +0000 UTC m=+68.125780307" watchObservedRunningTime="2026-05-28 19:44:14.838922326 +0000 UTC m=+68.126263231"
May 28 19:44:14.865031 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.864978    2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:14.868178 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:14.868139    2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:15.813828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:15.813775    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxzjd" event={"ID":"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc","Type":"ContainerStarted","Data":"4df47a513da81d24598bed0bf309397f2bb3449503a843ca801d8c79ce85bdc9"}
May 28 19:44:15.813828 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:15.813833    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-wxzjd" event={"ID":"0f40971c-dd1a-4403-a51b-bdd13d1fb3cc","Type":"ContainerStarted","Data":"bdc71ece0fb18db262756e102eace2a3b490e6511baea5af0c74505c6c3d4e02"}
May 28 19:44:15.814635 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:15.814605    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:15.816281 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:15.816249    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58c59b4dd8-8lv8f"
May 28 19:44:15.831862 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:15.831809    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-wxzjd" podStartSLOduration=67.184296652 podStartE2EDuration="1m8.83179324s" podCreationTimestamp="2026-05-28 19:43:07 +0000 UTC" firstStartedPulling="2026-05-28 19:44:13.500333013 +0000 UTC m=+66.787673895" lastFinishedPulling="2026-05-28 19:44:15.147829601 +0000 UTC m=+68.435170483" observedRunningTime="2026-05-28 19:44:15.831402664 +0000 UTC m=+69.118743574" watchObservedRunningTime="2026-05-28 19:44:15.83179324 +0000 UTC m=+69.119134144"
May 28 19:44:17.333349 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:17.333323    2568 scope.go:117] "RemoveContainer" containerID="f67f20af94f5fef2c8732956da853eb474e7847d78bc315b1f4f1f7a25e65d83"
May 28 19:44:18.826233 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.826162    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5" event={"ID":"4e5384f0-d883-464f-8ac5-02cbdd363eb4","Type":"ContainerStarted","Data":"9440750976b7dd3a42cc42ee570596323d40f45cd0feb253efb96e4f2a638fa4"}
May 28 19:44:18.826233 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.826209    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5" event={"ID":"4e5384f0-d883-464f-8ac5-02cbdd363eb4","Type":"ContainerStarted","Data":"14c777976ebc8375957750acff5601256800990942e8178c32184f2c050711a8"}
May 28 19:44:18.827461 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.827433    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvcmf" event={"ID":"0c57594e-7784-4885-ab2f-577790157a3f","Type":"ContainerStarted","Data":"ca706bdaf0aef3ea452929baa1d675c7784e5fa35d3accd8c0d80c913ec6082f"}
May 28 19:44:18.830056 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.830032    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-489wq" event={"ID":"d9ecf972-8310-46b3-93b8-37284fcd1e04","Type":"ContainerStarted","Data":"4d659eb3f222f6ee2d4f62668bd7ee51a89577c706d8ea0d511bb2184a3d417f"}
May 28 19:44:18.830193 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.830061    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-489wq" event={"ID":"d9ecf972-8310-46b3-93b8-37284fcd1e04","Type":"ContainerStarted","Data":"56ba9996152204e03d3184315dba6002ca3dcc73448733f4e784a96b053bd138"}
May 28 19:44:18.830545 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.830514    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-489wq"
May 28 19:44:18.831819 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.831795    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf" event={"ID":"561fea3e-dfb5-42b2-b1d1-0581fec1d6eb","Type":"ContainerStarted","Data":"700d2982b1f2a5d0dd149aa166ff9b83e0f3f8bd414cf86d6b7b69318567e427"}
May 28 19:44:18.833920 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.833902    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:44:18.834046 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.833968    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb" event={"ID":"5dc02322-296b-443f-bb2b-900dbdd4a4c8","Type":"ContainerStarted","Data":"5f7ff398e6598f6edbff2c96148b25dd881732fe22a5484c717851ac0c820f44"}
May 28 19:44:18.834247 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.834226    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:44:18.843285 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.843232    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-c8pj5" podStartSLOduration=50.392259746 podStartE2EDuration="54.843214182s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:44:13.967807728 +0000 UTC m=+67.255148610" lastFinishedPulling="2026-05-28 19:44:18.418762152 +0000 UTC m=+71.706103046" observedRunningTime="2026-05-28 19:44:18.843006389 +0000 UTC m=+72.130347294" watchObservedRunningTime="2026-05-28 19:44:18.843214182 +0000 UTC m=+72.130555089"
May 28 19:44:18.861661 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.861556    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-489wq" podStartSLOduration=33.537488547 podStartE2EDuration="37.861540468s" podCreationTimestamp="2026-05-28 19:43:41 +0000 UTC" firstStartedPulling="2026-05-28 19:44:14.094694432 +0000 UTC m=+67.382035314" lastFinishedPulling="2026-05-28 19:44:18.41874635 +0000 UTC m=+71.706087235" observedRunningTime="2026-05-28 19:44:18.860610286 +0000 UTC m=+72.147951190" watchObservedRunningTime="2026-05-28 19:44:18.861540468 +0000 UTC m=+72.148881377"
May 28 19:44:18.898919 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.898859    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-mq7cf" podStartSLOduration=50.480201057 podStartE2EDuration="54.898844159s" podCreationTimestamp="2026-05-28 19:43:24 +0000 UTC" firstStartedPulling="2026-05-28 19:44:14.001117784 +0000 UTC m=+67.288458666" lastFinishedPulling="2026-05-28 19:44:18.419760882 +0000 UTC m=+71.707101768" observedRunningTime="2026-05-28 19:44:18.895958102 +0000 UTC m=+72.183299005" watchObservedRunningTime="2026-05-28 19:44:18.898844159 +0000 UTC m=+72.186185062"
May 28 19:44:18.912995 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:18.912943    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lvcmf" podStartSLOduration=33.623152305 podStartE2EDuration="37.91292604s" podCreationTimestamp="2026-05-28 19:43:41 +0000 UTC" firstStartedPulling="2026-05-28 19:44:14.135105536 +0000 UTC m=+67.422446435" lastFinishedPulling="2026-05-28 19:44:18.424879275 +0000 UTC m=+71.712220170" observedRunningTime="2026-05-28 19:44:18.912191323 +0000 UTC m=+72.199532229" watchObservedRunningTime="2026-05-28 19:44:18.91292604 +0000 UTC m=+72.200266952"
May 28 19:44:19.360370 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:19.360338    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-9vlfb"
May 28 19:44:20.263333 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.263290    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lh4zd"]
May 28 19:44:20.267266 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.267233    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.270198 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.270163    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 19:44:20.270356 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.270310    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5qmfg\""
May 28 19:44:20.270555 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.270541    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 19:44:20.292458 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.292427    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lh4zd"]
May 28 19:44:20.405159 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.405117    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kksxq\" (UniqueName: \"kubernetes.io/projected/9336ef2c-04f1-4358-965f-12e45660a144-kube-api-access-kksxq\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.405159 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.405164    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9336ef2c-04f1-4358-965f-12e45660a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.405407 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.405186    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9336ef2c-04f1-4358-965f-12e45660a144-data-volume\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.405407 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.405307    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9336ef2c-04f1-4358-965f-12e45660a144-crio-socket\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.405407 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.405345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9336ef2c-04f1-4358-965f-12e45660a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.505852 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.505759    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9336ef2c-04f1-4358-965f-12e45660a144-crio-socket\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.505852 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.505796    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9336ef2c-04f1-4358-965f-12e45660a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.506072 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.505891    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9336ef2c-04f1-4358-965f-12e45660a144-crio-socket\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.506072 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.505928    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kksxq\" (UniqueName: \"kubernetes.io/projected/9336ef2c-04f1-4358-965f-12e45660a144-kube-api-access-kksxq\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.506072 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.505969    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9336ef2c-04f1-4358-965f-12e45660a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.506072 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.505996    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9336ef2c-04f1-4358-965f-12e45660a144-data-volume\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.506319 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.506303    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9336ef2c-04f1-4358-965f-12e45660a144-data-volume\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.506569 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.506553    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9336ef2c-04f1-4358-965f-12e45660a144-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.508218 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.508202    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9336ef2c-04f1-4358-965f-12e45660a144-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.517546 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.517520    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kksxq\" (UniqueName: \"kubernetes.io/projected/9336ef2c-04f1-4358-965f-12e45660a144-kube-api-access-kksxq\") pod \"insights-runtime-extractor-lh4zd\" (UID: \"9336ef2c-04f1-4358-965f-12e45660a144\") " pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.576926 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.576885    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lh4zd"
May 28 19:44:20.709274 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.709214    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lh4zd"]
May 28 19:44:20.713037 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:20.713005    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9336ef2c_04f1_4358_965f_12e45660a144.slice/crio-0394b6199656576a97733827dae603f01524f7569af8f33e26f2458eacd70ce8 WatchSource:0}: Error finding container 0394b6199656576a97733827dae603f01524f7569af8f33e26f2458eacd70ce8: Status 404 returned error can't find the container with id 0394b6199656576a97733827dae603f01524f7569af8f33e26f2458eacd70ce8
May 28 19:44:20.844242 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.844205    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh4zd" event={"ID":"9336ef2c-04f1-4358-965f-12e45660a144","Type":"ContainerStarted","Data":"2c4b3705f641f510a583b9f1d9960de247eedf97f6b1165122b92c0463a36cbd"}
May 28 19:44:20.844242 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:20.844242    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh4zd" event={"ID":"9336ef2c-04f1-4358-965f-12e45660a144","Type":"ContainerStarted","Data":"0394b6199656576a97733827dae603f01524f7569af8f33e26f2458eacd70ce8"}
May 28 19:44:21.850379 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:21.850335    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh4zd" event={"ID":"9336ef2c-04f1-4358-965f-12e45660a144","Type":"ContainerStarted","Data":"e039d9835c2eafc347af9058fbe90a03b910f6288a57000ce103de9d2cc5b0f0"}
May 28 19:44:23.858693 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:23.858659    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lh4zd" event={"ID":"9336ef2c-04f1-4358-965f-12e45660a144","Type":"ContainerStarted","Data":"d05267611b403c4d5417d867ca2e9a00fe612310ea267a1796d3efe3c583f21f"}
May 28 19:44:23.878227 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:23.878162    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lh4zd" podStartSLOduration=1.837132989 podStartE2EDuration="3.878144664s" podCreationTimestamp="2026-05-28 19:44:20 +0000 UTC" firstStartedPulling="2026-05-28 19:44:20.773147655 +0000 UTC m=+74.060488540" lastFinishedPulling="2026-05-28 19:44:22.814159333 +0000 UTC m=+76.101500215" observedRunningTime="2026-05-28 19:44:23.876996811 +0000 UTC m=+77.164337716" watchObservedRunningTime="2026-05-28 19:44:23.878144664 +0000 UTC m=+77.165485568"
May 28 19:44:29.847400 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:29.847285    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-489wq"
May 28 19:44:33.501203 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.501167    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mqjcz"]
May 28 19:44:33.504255 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.504233    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.506648 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.506601    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 19:44:33.506782 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.506744    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 19:44:33.507575 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.507560    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nnht5\""
May 28 19:44:33.507632 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.507599    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 19:44:33.508069 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.508054    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 19:44:33.600202 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600153    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-tls\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600202 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600215    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-textfile\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600234    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9c58404-d17e-42d4-9d1e-982278a6606a-metrics-client-ca\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600257    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rtg\" (UniqueName: \"kubernetes.io/projected/d9c58404-d17e-42d4-9d1e-982278a6606a-kube-api-access-h6rtg\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600281    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600337    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-sys\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600357    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-wtmp\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.600429 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.600424    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-root\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.700942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.700896    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-textfile\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.700942 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.700940    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9c58404-d17e-42d4-9d1e-982278a6606a-metrics-client-ca\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701147 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701062    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rtg\" (UniqueName: \"kubernetes.io/projected/d9c58404-d17e-42d4-9d1e-982278a6606a-kube-api-access-h6rtg\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701147 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701114    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701214 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701160    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-sys\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701214 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701187    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701314 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701230    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-wtmp\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701314 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701258    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-root\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701314 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701274    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-sys\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701459 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701335    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-textfile\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701459 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701341    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-root\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701459 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701357    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-tls\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701667 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701448    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-wtmp\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701667 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701568    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9c58404-d17e-42d4-9d1e-982278a6606a-metrics-client-ca\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.701758 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.701725    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.703676 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.703653    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.703771 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.703743    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9c58404-d17e-42d4-9d1e-982278a6606a-node-exporter-tls\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.712881 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.712853    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rtg\" (UniqueName: \"kubernetes.io/projected/d9c58404-d17e-42d4-9d1e-982278a6606a-kube-api-access-h6rtg\") pod \"node-exporter-mqjcz\" (UID: \"d9c58404-d17e-42d4-9d1e-982278a6606a\") " pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.809300 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.809260    2568 patch_prober.go:28] interesting pod/image-registry-b4944bb8f-8m9c7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 28 19:44:33.809461 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.809335    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" podUID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 19:44:33.814171 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.814147    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mqjcz"
May 28 19:44:33.824445 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:44:33.824399    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c58404_d17e_42d4_9d1e_982278a6606a.slice/crio-7c6e71e468089dc6a5158b4ead9bf5d5a5cd1a56ae9fe04fa52fdc724cd80e4a WatchSource:0}: Error finding container 7c6e71e468089dc6a5158b4ead9bf5d5a5cd1a56ae9fe04fa52fdc724cd80e4a: Status 404 returned error can't find the container with id 7c6e71e468089dc6a5158b4ead9bf5d5a5cd1a56ae9fe04fa52fdc724cd80e4a
May 28 19:44:33.893995 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:33.893958    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mqjcz" event={"ID":"d9c58404-d17e-42d4-9d1e-982278a6606a","Type":"ContainerStarted","Data":"7c6e71e468089dc6a5158b4ead9bf5d5a5cd1a56ae9fe04fa52fdc724cd80e4a"}
May 28 19:44:35.820138 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:35.820105    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:44:35.901623 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:35.901587    2568 generic.go:358] "Generic (PLEG): container finished" podID="d9c58404-d17e-42d4-9d1e-982278a6606a" containerID="bc336d2340d642637af195bb6091d61aad63171de01078ddc00213583007b662" exitCode=0
May 28 19:44:35.901788 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:35.901670    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mqjcz" event={"ID":"d9c58404-d17e-42d4-9d1e-982278a6606a","Type":"ContainerDied","Data":"bc336d2340d642637af195bb6091d61aad63171de01078ddc00213583007b662"}
May 28 19:44:36.905990 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:36.905955    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mqjcz" event={"ID":"d9c58404-d17e-42d4-9d1e-982278a6606a","Type":"ContainerStarted","Data":"1d41229e4f0cfaff121e1794323a45215ae810998a3a3ee5e60671d657d979cf"}
May 28 19:44:36.905990 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:36.905991    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mqjcz" event={"ID":"d9c58404-d17e-42d4-9d1e-982278a6606a","Type":"ContainerStarted","Data":"92c783278ad0a35f0e125814182b9e894bf8e5046f1723262f67d334f39f215b"}
May 28 19:44:36.948278 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:36.948210    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mqjcz" podStartSLOduration=2.860910327 podStartE2EDuration="3.948189074s" podCreationTimestamp="2026-05-28 19:44:33 +0000 UTC" firstStartedPulling="2026-05-28 19:44:33.826360994 +0000 UTC m=+87.113701876" lastFinishedPulling="2026-05-28 19:44:34.913639741 +0000 UTC m=+88.200980623" observedRunningTime="2026-05-28 19:44:36.947513842 +0000 UTC m=+90.234854746" watchObservedRunningTime="2026-05-28 19:44:36.948189074 +0000 UTC m=+90.235529982"
May 28 19:44:42.392514 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:42.392458    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b4944bb8f-8m9c7"]
May 28 19:44:45.817156 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:44:45.817122    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r9l4n"
May 28 19:45:07.412568 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.412517    2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" podUID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" containerName="registry" containerID="cri-o://fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3" gracePeriod=30
May 28 19:45:07.658869 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.658840    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:45:07.811512 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811455    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-installation-pull-secrets\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811550    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-bound-sa-token\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811585    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-image-registry-private-configuration\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811649 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811619    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811765 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811649    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-certificates\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811765 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811707    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdmcv\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-kube-api-access-hdmcv\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811765 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811737    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-trusted-ca\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.811908 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.811793    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a5f910-b419-4fde-afac-6e0adc35d8a4-ca-trust-extracted\") pod \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\" (UID: \"f8a5f910-b419-4fde-afac-6e0adc35d8a4\") "
May 28 19:45:07.812276 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.812241    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 19:45:07.812424 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.812314    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 19:45:07.814585 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.814544    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-kube-api-access-hdmcv" (OuterVolumeSpecName: "kube-api-access-hdmcv") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "kube-api-access-hdmcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 19:45:07.814692 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.814601    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 19:45:07.814692 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.814651    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 19:45:07.814822 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.814721    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 19:45:07.814822 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.814805    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 19:45:07.820721 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.820696    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a5f910-b419-4fde-afac-6e0adc35d8a4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f8a5f910-b419-4fde-afac-6e0adc35d8a4" (UID: "f8a5f910-b419-4fde-afac-6e0adc35d8a4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 19:45:07.913151 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913110    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a5f910-b419-4fde-afac-6e0adc35d8a4-ca-trust-extracted\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913151 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913141    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-installation-pull-secrets\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913151 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913151    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-bound-sa-token\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913369 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913162    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a5f910-b419-4fde-afac-6e0adc35d8a4-image-registry-private-configuration\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913369 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913172    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-tls\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913369 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913180    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-registry-certificates\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913369 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913191    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdmcv\" (UniqueName: \"kubernetes.io/projected/f8a5f910-b419-4fde-afac-6e0adc35d8a4-kube-api-access-hdmcv\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.913369 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.913199    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a5f910-b419-4fde-afac-6e0adc35d8a4-trusted-ca\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:45:07.994973 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.994937    2568 generic.go:358] "Generic (PLEG): container finished" podID="168f7593-5ec2-4f21-9aa9-c4e4cebe52f5" containerID="54e01316997f205f143eb09cdc83978bbf82441142543f99fc2ba17b6ecfcdc6" exitCode=0
May 28 19:45:07.995138 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.995015    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt" event={"ID":"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5","Type":"ContainerDied","Data":"54e01316997f205f143eb09cdc83978bbf82441142543f99fc2ba17b6ecfcdc6"}
May 28 19:45:07.995409 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.995389    2568 scope.go:117] "RemoveContainer" containerID="54e01316997f205f143eb09cdc83978bbf82441142543f99fc2ba17b6ecfcdc6"
May 28 19:45:07.996269 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.996252    2568 generic.go:358] "Generic (PLEG): container finished" podID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" containerID="fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3" exitCode=0
May 28 19:45:07.996325 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.996308    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7"
May 28 19:45:07.996360 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.996335    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" event={"ID":"f8a5f910-b419-4fde-afac-6e0adc35d8a4","Type":"ContainerDied","Data":"fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3"}
May 28 19:45:07.996394 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.996367    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b4944bb8f-8m9c7" event={"ID":"f8a5f910-b419-4fde-afac-6e0adc35d8a4","Type":"ContainerDied","Data":"f4e330bbf6511605772290b1340826045b7a1e9fe1f92dcad615ac112a79fa5f"}
May 28 19:45:07.996394 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:07.996386    2568 scope.go:117] "RemoveContainer" containerID="fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3"
May 28 19:45:08.006103 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:08.006080    2568 scope.go:117] "RemoveContainer" containerID="fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3"
May 28 19:45:08.006410 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:45:08.006389    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3\": container with ID starting with fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3 not found: ID does not exist" containerID="fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3"
May 28 19:45:08.006493 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:08.006419    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3"} err="failed to get container status \"fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3\": rpc error: code = NotFound desc = could not find container \"fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3\": container with ID starting with fd5ed44a0977e530df66a8523945af6d0f2a1240e5e209bfa83a31e26184b7d3 not found: ID does not exist"
May 28 19:45:08.026896 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:08.026866    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b4944bb8f-8m9c7"]
May 28 19:45:08.032578 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:08.032533    2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b4944bb8f-8m9c7"]
May 28 19:45:09.000945 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:09.000909    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-kx6jt" event={"ID":"168f7593-5ec2-4f21-9aa9-c4e4cebe52f5","Type":"ContainerStarted","Data":"19aca8e6ded4efb914581fa65e99d6c37478eda7ae418b0e994e19a5445f679d"}
May 28 19:45:09.334163 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:09.334125    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" path="/var/lib/kubelet/pods/f8a5f910-b419-4fde-afac-6e0adc35d8a4/volumes"
May 28 19:45:19.035237 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:19.035202    2568 generic.go:358] "Generic (PLEG): container finished" podID="63e20a96-5b18-4200-99ee-4d2d35606b95" containerID="a4b197dcfb4684aae81cbd63e09898a9754917441de7e021d8da9113c6c6bcbe" exitCode=0
May 28 19:45:19.035661 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:19.035249    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-2t85s" event={"ID":"63e20a96-5b18-4200-99ee-4d2d35606b95","Type":"ContainerDied","Data":"a4b197dcfb4684aae81cbd63e09898a9754917441de7e021d8da9113c6c6bcbe"}
May 28 19:45:19.035661 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:19.035587    2568 scope.go:117] "RemoveContainer" containerID="a4b197dcfb4684aae81cbd63e09898a9754917441de7e021d8da9113c6c6bcbe"
May 28 19:45:20.039308 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:20.039268    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-2t85s" event={"ID":"63e20a96-5b18-4200-99ee-4d2d35606b95","Type":"ContainerStarted","Data":"c76fad1b6a833cebdf9a9c80c0b99ab061d6b3eb3b060f1b069b0ec082319ccf"}
May 28 19:45:22.163508 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:22.163440    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" podUID="bb42dec1-4f99-4d22-a322-26acf79d0d47" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 19:45:32.163448 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:32.163406    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" podUID="bb42dec1-4f99-4d22-a322-26acf79d0d47" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 19:45:33.077916 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:33.077869    2568 generic.go:358] "Generic (PLEG): container finished" podID="bda09b67-e01f-4af9-91a4-6aa913ab86d8" containerID="a1e8cce90d4642ba45594ba812f435ee53ef149ebc9e34f5fe7b109584e63549" exitCode=0
May 28 19:45:33.078113 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:33.077942    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46" event={"ID":"bda09b67-e01f-4af9-91a4-6aa913ab86d8","Type":"ContainerDied","Data":"a1e8cce90d4642ba45594ba812f435ee53ef149ebc9e34f5fe7b109584e63549"}
May 28 19:45:33.078337 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:33.078321    2568 scope.go:117] "RemoveContainer" containerID="a1e8cce90d4642ba45594ba812f435ee53ef149ebc9e34f5fe7b109584e63549"
May 28 19:45:34.084434 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:34.084396    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-r5f46" event={"ID":"bda09b67-e01f-4af9-91a4-6aa913ab86d8","Type":"ContainerStarted","Data":"c3082880f45e32fee92c7206fb01345da534f471fb4bb392af68efae32365e18"}
May 28 19:45:42.163693 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:42.163652    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" podUID="bb42dec1-4f99-4d22-a322-26acf79d0d47" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
May 28 19:45:42.164087 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:42.163737    2568 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm"
May 28 19:45:42.164243 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:42.164222    2568 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"317a979b7e646851fa4b8ffd003f67dcb982807317aedce55accaa15e4a63bd5"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" containerMessage="Container service-proxy failed liveness probe, will be restarted"
May 28 19:45:42.164279 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:42.164263    2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" podUID="bb42dec1-4f99-4d22-a322-26acf79d0d47" containerName="service-proxy" containerID="cri-o://317a979b7e646851fa4b8ffd003f67dcb982807317aedce55accaa15e4a63bd5" gracePeriod=30
May 28 19:45:43.114182 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:43.114144    2568 generic.go:358] "Generic (PLEG): container finished" podID="bb42dec1-4f99-4d22-a322-26acf79d0d47" containerID="317a979b7e646851fa4b8ffd003f67dcb982807317aedce55accaa15e4a63bd5" exitCode=2
May 28 19:45:43.114363 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:43.114193    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" event={"ID":"bb42dec1-4f99-4d22-a322-26acf79d0d47","Type":"ContainerDied","Data":"317a979b7e646851fa4b8ffd003f67dcb982807317aedce55accaa15e4a63bd5"}
May 28 19:45:43.114363 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:45:43.114229    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-685496b79d-nmlrm" event={"ID":"bb42dec1-4f99-4d22-a322-26acf79d0d47","Type":"ContainerStarted","Data":"4fc232824310ea4b8d45b140d664b48cac4b8f5e9d192115f4f001354138b60b"}
May 28 19:48:02.141214 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.141167    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"]
May 28 19:48:02.141702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.141586    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" containerName="registry"
May 28 19:48:02.141702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.141601    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" containerName="registry"
May 28 19:48:02.141702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.141693    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="f8a5f910-b419-4fde-afac-6e0adc35d8a4" containerName="registry"
May 28 19:48:02.144723 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.144705    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.147228 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.147202    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jbt6w\""
May 28 19:48:02.147368 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.147203    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 28 19:48:02.147487 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.147457    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 28 19:48:02.147662 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.147650    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 28 19:48:02.147705 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.147682    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 28 19:48:02.160767 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.160737    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"]
May 28 19:48:02.184236 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.184202    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/752b8b27-f963-45a7-b7fd-d677aba0da51-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.184418 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.184250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/752b8b27-f963-45a7-b7fd-d677aba0da51-webhook-cert\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.184418 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.184296    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngzbp\" (UniqueName: \"kubernetes.io/projected/752b8b27-f963-45a7-b7fd-d677aba0da51-kube-api-access-ngzbp\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.284721 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.284675    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/752b8b27-f963-45a7-b7fd-d677aba0da51-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.284721 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.284730    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/752b8b27-f963-45a7-b7fd-d677aba0da51-webhook-cert\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.284987 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.284755    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ngzbp\" (UniqueName: \"kubernetes.io/projected/752b8b27-f963-45a7-b7fd-d677aba0da51-kube-api-access-ngzbp\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.287581 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.287548    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/752b8b27-f963-45a7-b7fd-d677aba0da51-webhook-cert\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.287703 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.287548    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/752b8b27-f963-45a7-b7fd-d677aba0da51-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.294055 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.294022    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngzbp\" (UniqueName: \"kubernetes.io/projected/752b8b27-f963-45a7-b7fd-d677aba0da51-kube-api-access-ngzbp\") pod \"opendatahub-operator-controller-manager-69568b6749-c285w\" (UID: \"752b8b27-f963-45a7-b7fd-d677aba0da51\") " pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.456364 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.456258    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:02.600638 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:02.600601    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"]
May 28 19:48:02.604737 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:48:02.604702    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752b8b27_f963_45a7_b7fd_d677aba0da51.slice/crio-a1e82f5a0a9ae5049912bad98a7e0f97071e8fe50eaff03eca56cf0c1f5c6729 WatchSource:0}: Error finding container a1e82f5a0a9ae5049912bad98a7e0f97071e8fe50eaff03eca56cf0c1f5c6729: Status 404 returned error can't find the container with id a1e82f5a0a9ae5049912bad98a7e0f97071e8fe50eaff03eca56cf0c1f5c6729
May 28 19:48:03.516686 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:03.516653    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w" event={"ID":"752b8b27-f963-45a7-b7fd-d677aba0da51","Type":"ContainerStarted","Data":"a1e82f5a0a9ae5049912bad98a7e0f97071e8fe50eaff03eca56cf0c1f5c6729"}
May 28 19:48:05.525310 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:05.525265    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w" event={"ID":"752b8b27-f963-45a7-b7fd-d677aba0da51","Type":"ContainerStarted","Data":"f23cb9a158b7587995dced44b558ea997928f2e3c118d82159a1517a6d893991"}
May 28 19:48:05.525751 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:05.525452    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:05.554768 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:05.554718    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w" podStartSLOduration=0.811899039 podStartE2EDuration="3.554699194s" podCreationTimestamp="2026-05-28 19:48:02 +0000 UTC" firstStartedPulling="2026-05-28 19:48:02.606269141 +0000 UTC m=+295.893610023" lastFinishedPulling="2026-05-28 19:48:05.349069296 +0000 UTC m=+298.636410178" observedRunningTime="2026-05-28 19:48:05.553421215 +0000 UTC m=+298.840762155" watchObservedRunningTime="2026-05-28 19:48:05.554699194 +0000 UTC m=+298.842040098"
May 28 19:48:07.237150 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:07.237122    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:48:07.237575 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:07.237326    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:48:07.242290 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:07.242254    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:48:07.243003 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:07.242981    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:48:07.249765 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:07.249727    2568 kubelet.go:1628] "Image garbage collection succeeded"
May 28 19:48:16.530863 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:16.530821    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69568b6749-c285w"
May 28 19:48:20.688406 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.688370    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58474d499-pvr2w"]
May 28 19:48:20.691511 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.691492    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.698311 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.697985    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
May 28 19:48:20.698311 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.698050    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pqf79\""
May 28 19:48:20.698311 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.698056    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
May 28 19:48:20.715412 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.715375    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58474d499-pvr2w"]
May 28 19:48:20.835447 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.835408    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b40d81b-7ffb-4649-9271-eb495e680d77-tmp\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.835447 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.835449    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b40d81b-7ffb-4649-9271-eb495e680d77-tls-certs\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.835707 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.835497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k7p4\" (UniqueName: \"kubernetes.io/projected/1b40d81b-7ffb-4649-9271-eb495e680d77-kube-api-access-8k7p4\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.936433 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.936394    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b40d81b-7ffb-4649-9271-eb495e680d77-tmp\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.936433 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.936435    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b40d81b-7ffb-4649-9271-eb495e680d77-tls-certs\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.936721 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.936454    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8k7p4\" (UniqueName: \"kubernetes.io/projected/1b40d81b-7ffb-4649-9271-eb495e680d77-kube-api-access-8k7p4\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.938885 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.938824    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b40d81b-7ffb-4649-9271-eb495e680d77-tmp\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.939022 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.939002    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b40d81b-7ffb-4649-9271-eb495e680d77-tls-certs\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:20.945720 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:20.945689    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k7p4\" (UniqueName: \"kubernetes.io/projected/1b40d81b-7ffb-4649-9271-eb495e680d77-kube-api-access-8k7p4\") pod \"kube-auth-proxy-58474d499-pvr2w\" (UID: \"1b40d81b-7ffb-4649-9271-eb495e680d77\") " pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:21.001701 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:21.001647    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w"
May 28 19:48:21.130557 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:21.130522    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58474d499-pvr2w"]
May 28 19:48:21.133366 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:48:21.133339    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b40d81b_7ffb_4649_9271_eb495e680d77.slice/crio-c4e7a46e3024b494df2706cf93a46cf320fff34a473e051a1364775c1355b88f WatchSource:0}: Error finding container c4e7a46e3024b494df2706cf93a46cf320fff34a473e051a1364775c1355b88f: Status 404 returned error can't find the container with id c4e7a46e3024b494df2706cf93a46cf320fff34a473e051a1364775c1355b88f
May 28 19:48:21.135112 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:21.135093    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 19:48:21.574386 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:21.574347    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w" event={"ID":"1b40d81b-7ffb-4649-9271-eb495e680d77","Type":"ContainerStarted","Data":"c4e7a46e3024b494df2706cf93a46cf320fff34a473e051a1364775c1355b88f"}
May 28 19:48:24.413046 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.413008    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-rjql7"]
May 28 19:48:24.416134 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.416118    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:24.418581 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.418553    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
May 28 19:48:24.418715 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.418620    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-fjpjb\""
May 28 19:48:24.424365 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.424341    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-rjql7"]
May 28 19:48:24.468972 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.468936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct89r\" (UniqueName: \"kubernetes.io/projected/87f1d890-260d-41b4-8453-2874a5269507-kube-api-access-ct89r\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:24.469133 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.469040    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f1d890-260d-41b4-8453-2874a5269507-cert\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:24.570480 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.570427    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ct89r\" (UniqueName: \"kubernetes.io/projected/87f1d890-260d-41b4-8453-2874a5269507-kube-api-access-ct89r\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:24.570655 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.570507    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f1d890-260d-41b4-8453-2874a5269507-cert\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:24.570655 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:24.570634    2568 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
May 28 19:48:24.570736 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:24.570681    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f1d890-260d-41b4-8453-2874a5269507-cert podName:87f1d890-260d-41b4-8453-2874a5269507 nodeName:}" failed. No retries permitted until 2026-05-28 19:48:25.070664824 +0000 UTC m=+318.358005706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/87f1d890-260d-41b4-8453-2874a5269507-cert") pod "odh-model-controller-858dbf95b8-rjql7" (UID: "87f1d890-260d-41b4-8453-2874a5269507") : secret "odh-model-controller-webhook-cert" not found
May 28 19:48:24.585451 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.585418    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct89r\" (UniqueName: \"kubernetes.io/projected/87f1d890-260d-41b4-8453-2874a5269507-kube-api-access-ct89r\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:24.586523 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.586492    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w" event={"ID":"1b40d81b-7ffb-4649-9271-eb495e680d77","Type":"ContainerStarted","Data":"6cd33cf3696b19777d7f9830530c65c570f41f9a1afe5f992fc80038c2f83498"}
May 28 19:48:24.616066 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:24.616012    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58474d499-pvr2w" podStartSLOduration=2.056418113 podStartE2EDuration="4.615997314s" podCreationTimestamp="2026-05-28 19:48:20 +0000 UTC" firstStartedPulling="2026-05-28 19:48:21.135245355 +0000 UTC m=+314.422586237" lastFinishedPulling="2026-05-28 19:48:23.694824542 +0000 UTC m=+316.982165438" observedRunningTime="2026-05-28 19:48:24.614776955 +0000 UTC m=+317.902117860" watchObservedRunningTime="2026-05-28 19:48:24.615997314 +0000 UTC m=+317.903338218"
May 28 19:48:25.075336 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:25.075289    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f1d890-260d-41b4-8453-2874a5269507-cert\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:25.077960 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:25.077933    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/87f1d890-260d-41b4-8453-2874a5269507-cert\") pod \"odh-model-controller-858dbf95b8-rjql7\" (UID: \"87f1d890-260d-41b4-8453-2874a5269507\") " pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:25.326256 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:25.326171    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:25.460132 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:25.460106    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-rjql7"]
May 28 19:48:25.462770 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:48:25.462737    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f1d890_260d_41b4_8453_2874a5269507.slice/crio-ab05c3304a6bbf42496f9f933f8fc319a3272d4f90e51aa8e11ac0a3f741975a WatchSource:0}: Error finding container ab05c3304a6bbf42496f9f933f8fc319a3272d4f90e51aa8e11ac0a3f741975a: Status 404 returned error can't find the container with id ab05c3304a6bbf42496f9f933f8fc319a3272d4f90e51aa8e11ac0a3f741975a
May 28 19:48:25.590892 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:25.590803    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" event={"ID":"87f1d890-260d-41b4-8453-2874a5269507","Type":"ContainerStarted","Data":"ab05c3304a6bbf42496f9f933f8fc319a3272d4f90e51aa8e11ac0a3f741975a"}
May 28 19:48:28.603262 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:28.603172    2568 generic.go:358] "Generic (PLEG): container finished" podID="87f1d890-260d-41b4-8453-2874a5269507" containerID="131062814f6a5ebdb0cc92923c50195679ef39327bfb88340c654086f2ab0fc5" exitCode=1
May 28 19:48:28.603262 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:28.603222    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" event={"ID":"87f1d890-260d-41b4-8453-2874a5269507","Type":"ContainerDied","Data":"131062814f6a5ebdb0cc92923c50195679ef39327bfb88340c654086f2ab0fc5"}
May 28 19:48:28.603670 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:28.603464    2568 scope.go:117] "RemoveContainer" containerID="131062814f6a5ebdb0cc92923c50195679ef39327bfb88340c654086f2ab0fc5"
May 28 19:48:29.608056 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:29.608015    2568 generic.go:358] "Generic (PLEG): container finished" podID="87f1d890-260d-41b4-8453-2874a5269507" containerID="d343a4ff409f1e159698059e4629127866aefaaf5385a99883ed808a48deb2c9" exitCode=1
May 28 19:48:29.608515 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:29.608103    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" event={"ID":"87f1d890-260d-41b4-8453-2874a5269507","Type":"ContainerDied","Data":"d343a4ff409f1e159698059e4629127866aefaaf5385a99883ed808a48deb2c9"}
May 28 19:48:29.608515 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:29.608153    2568 scope.go:117] "RemoveContainer" containerID="131062814f6a5ebdb0cc92923c50195679ef39327bfb88340c654086f2ab0fc5"
May 28 19:48:29.608515 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:29.608368    2568 scope.go:117] "RemoveContainer" containerID="d343a4ff409f1e159698059e4629127866aefaaf5385a99883ed808a48deb2c9"
May 28 19:48:29.608632 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:29.608601    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-rjql7_opendatahub(87f1d890-260d-41b4-8453-2874a5269507)\"" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" podUID="87f1d890-260d-41b4-8453-2874a5269507"
May 28 19:48:30.511963 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.511924    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-tfpxs"]
May 28 19:48:30.516437 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.516403    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:30.518965 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.518941    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6vd6r\""
May 28 19:48:30.519086 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.518941    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
May 28 19:48:30.528046 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.528014    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-tfpxs"]
May 28 19:48:30.612554 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.612527    2568 scope.go:117] "RemoveContainer" containerID="d343a4ff409f1e159698059e4629127866aefaaf5385a99883ed808a48deb2c9"
May 28 19:48:30.612922 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:30.612714    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-rjql7_opendatahub(87f1d890-260d-41b4-8453-2874a5269507)\"" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" podUID="87f1d890-260d-41b4-8453-2874a5269507"
May 28 19:48:30.624593 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.624563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655333f5-f77e-4b0d-b0a1-2381f51373c8-cert\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:30.624739 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.624605    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5px5\" (UniqueName: \"kubernetes.io/projected/655333f5-f77e-4b0d-b0a1-2381f51373c8-kube-api-access-s5px5\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:30.725223 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.725173    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655333f5-f77e-4b0d-b0a1-2381f51373c8-cert\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:30.725411 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.725238    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s5px5\" (UniqueName: \"kubernetes.io/projected/655333f5-f77e-4b0d-b0a1-2381f51373c8-kube-api-access-s5px5\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:30.725411 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:30.725345    2568 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
May 28 19:48:30.725536 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:30.725435    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/655333f5-f77e-4b0d-b0a1-2381f51373c8-cert podName:655333f5-f77e-4b0d-b0a1-2381f51373c8 nodeName:}" failed. No retries permitted until 2026-05-28 19:48:31.225409221 +0000 UTC m=+324.512750116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/655333f5-f77e-4b0d-b0a1-2381f51373c8-cert") pod "kserve-controller-manager-856948b99f-tfpxs" (UID: "655333f5-f77e-4b0d-b0a1-2381f51373c8") : secret "kserve-webhook-server-cert" not found
May 28 19:48:30.734999 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:30.734967    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5px5\" (UniqueName: \"kubernetes.io/projected/655333f5-f77e-4b0d-b0a1-2381f51373c8-kube-api-access-s5px5\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:31.230548 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:31.230511    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655333f5-f77e-4b0d-b0a1-2381f51373c8-cert\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:31.233226 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:31.233193    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/655333f5-f77e-4b0d-b0a1-2381f51373c8-cert\") pod \"kserve-controller-manager-856948b99f-tfpxs\" (UID: \"655333f5-f77e-4b0d-b0a1-2381f51373c8\") " pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:31.429357 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:31.429317    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:31.567435 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:31.567399    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-tfpxs"]
May 28 19:48:31.570400 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:48:31.570374    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655333f5_f77e_4b0d_b0a1_2381f51373c8.slice/crio-70f36a8bce085e354fe5275092b65cc6a3489c7a67539467f1020aa7758b3fbf WatchSource:0}: Error finding container 70f36a8bce085e354fe5275092b65cc6a3489c7a67539467f1020aa7758b3fbf: Status 404 returned error can't find the container with id 70f36a8bce085e354fe5275092b65cc6a3489c7a67539467f1020aa7758b3fbf
May 28 19:48:31.616400 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:31.616367    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs" event={"ID":"655333f5-f77e-4b0d-b0a1-2381f51373c8","Type":"ContainerStarted","Data":"70f36a8bce085e354fe5275092b65cc6a3489c7a67539467f1020aa7758b3fbf"}
May 28 19:48:35.326660 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:35.326626    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:35.327082 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:35.327023    2568 scope.go:117] "RemoveContainer" containerID="d343a4ff409f1e159698059e4629127866aefaaf5385a99883ed808a48deb2c9"
May 28 19:48:35.327198 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:48:35.327181    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-rjql7_opendatahub(87f1d890-260d-41b4-8453-2874a5269507)\"" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" podUID="87f1d890-260d-41b4-8453-2874a5269507"
May 28 19:48:35.631880 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:35.631795    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs" event={"ID":"655333f5-f77e-4b0d-b0a1-2381f51373c8","Type":"ContainerStarted","Data":"3d15a4b499d108eba73c5b28b8da9e510f210863e9117f886bcbb1ab24a53927"}
May 28 19:48:35.632018 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:35.631899    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:48:35.671030 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:35.670975    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs" podStartSLOduration=2.57127304 podStartE2EDuration="5.670958018s" podCreationTimestamp="2026-05-28 19:48:30 +0000 UTC" firstStartedPulling="2026-05-28 19:48:31.571762401 +0000 UTC m=+324.859103286" lastFinishedPulling="2026-05-28 19:48:34.671447378 +0000 UTC m=+327.958788264" observedRunningTime="2026-05-28 19:48:35.669255207 +0000 UTC m=+328.956596122" watchObservedRunningTime="2026-05-28 19:48:35.670958018 +0000 UTC m=+328.958298921"
May 28 19:48:37.357755 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.357716    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"]
May 28 19:48:37.361279 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.361258    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.366078 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.366055    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\""
May 28 19:48:37.366078 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.366064    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-rjcwp\""
May 28 19:48:37.366269 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.366132    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\""
May 28 19:48:37.373641 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.373616    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"]
May 28 19:48:37.486487 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.486438    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/4cee6f3c-15a5-4967-9b22-fcd956164197-operator-config\") pod \"servicemesh-operator3-55f49c5f94-g7chn\" (UID: \"4cee6f3c-15a5-4967-9b22-fcd956164197\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.486688 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.486520    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gt8r\" (UniqueName: \"kubernetes.io/projected/4cee6f3c-15a5-4967-9b22-fcd956164197-kube-api-access-4gt8r\") pod \"servicemesh-operator3-55f49c5f94-g7chn\" (UID: \"4cee6f3c-15a5-4967-9b22-fcd956164197\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.587071 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.587032    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/4cee6f3c-15a5-4967-9b22-fcd956164197-operator-config\") pod \"servicemesh-operator3-55f49c5f94-g7chn\" (UID: \"4cee6f3c-15a5-4967-9b22-fcd956164197\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.587071 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.587072    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4gt8r\" (UniqueName: \"kubernetes.io/projected/4cee6f3c-15a5-4967-9b22-fcd956164197-kube-api-access-4gt8r\") pod \"servicemesh-operator3-55f49c5f94-g7chn\" (UID: \"4cee6f3c-15a5-4967-9b22-fcd956164197\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.589746 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.589711    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/4cee6f3c-15a5-4967-9b22-fcd956164197-operator-config\") pod \"servicemesh-operator3-55f49c5f94-g7chn\" (UID: \"4cee6f3c-15a5-4967-9b22-fcd956164197\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.604073 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.604039    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gt8r\" (UniqueName: \"kubernetes.io/projected/4cee6f3c-15a5-4967-9b22-fcd956164197-kube-api-access-4gt8r\") pod \"servicemesh-operator3-55f49c5f94-g7chn\" (UID: \"4cee6f3c-15a5-4967-9b22-fcd956164197\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.670461 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.670369    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:37.828559 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:37.828529    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"]
May 28 19:48:37.830311 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:48:37.830280    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cee6f3c_15a5_4967_9b22_fcd956164197.slice/crio-f4b891c26ba036151a30a35984a9c8bad316d5aea279791c1030c1d80a3ec9d2 WatchSource:0}: Error finding container f4b891c26ba036151a30a35984a9c8bad316d5aea279791c1030c1d80a3ec9d2: Status 404 returned error can't find the container with id f4b891c26ba036151a30a35984a9c8bad316d5aea279791c1030c1d80a3ec9d2
May 28 19:48:38.643340 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:38.643299    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn" event={"ID":"4cee6f3c-15a5-4967-9b22-fcd956164197","Type":"ContainerStarted","Data":"f4b891c26ba036151a30a35984a9c8bad316d5aea279791c1030c1d80a3ec9d2"}
May 28 19:48:41.656289 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:41.656253    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn" event={"ID":"4cee6f3c-15a5-4967-9b22-fcd956164197","Type":"ContainerStarted","Data":"a3bdf9b0e6dd27fe71d1b879c3432a045c0ad343b8d09e895d823db0eba69249"}
May 28 19:48:41.656705 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:41.656351    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:41.679027 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:41.678961    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn" podStartSLOduration=1.403435077 podStartE2EDuration="4.678939615s" podCreationTimestamp="2026-05-28 19:48:37 +0000 UTC" firstStartedPulling="2026-05-28 19:48:37.832888609 +0000 UTC m=+331.120229491" lastFinishedPulling="2026-05-28 19:48:41.108393147 +0000 UTC m=+334.395734029" observedRunningTime="2026-05-28 19:48:41.678296756 +0000 UTC m=+334.965637665" watchObservedRunningTime="2026-05-28 19:48:41.678939615 +0000 UTC m=+334.966280519"
May 28 19:48:45.326923 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:45.326862    2568 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:45.327325 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:45.327278    2568 scope.go:117] "RemoveContainer" containerID="d343a4ff409f1e159698059e4629127866aefaaf5385a99883ed808a48deb2c9"
May 28 19:48:46.674646 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.674607    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" event={"ID":"87f1d890-260d-41b4-8453-2874a5269507","Type":"ContainerStarted","Data":"450d15d91acdb724ac70b28201ec276baf636b7308996ccb76fa0e2bdc9ec638"}
May 28 19:48:46.675180 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.674842    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:48:46.694718 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.694655    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7" podStartSLOduration=2.510074158 podStartE2EDuration="22.694634251s" podCreationTimestamp="2026-05-28 19:48:24 +0000 UTC" firstStartedPulling="2026-05-28 19:48:25.464010414 +0000 UTC m=+318.751351297" lastFinishedPulling="2026-05-28 19:48:45.648570503 +0000 UTC m=+338.935911390" observedRunningTime="2026-05-28 19:48:46.69232588 +0000 UTC m=+339.979666783" watchObservedRunningTime="2026-05-28 19:48:46.694634251 +0000 UTC m=+339.981975155"
May 28 19:48:46.950799 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.950715    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"]
May 28 19:48:46.954079 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.954055    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.956685 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.956660    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
May 28 19:48:46.956829 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.956736    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
May 28 19:48:46.956829 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.956741    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
May 28 19:48:46.956829 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.956740    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nlww9\""
May 28 19:48:46.957005 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.956660    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 28 19:48:46.967562 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967535    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-local-certs\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.967702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967577    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.967702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.967702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967657    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5v2x\" (UniqueName: \"kubernetes.io/projected/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-kube-api-access-l5v2x\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.967702 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-token\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.967845 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967719    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-cacerts\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.967845 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.967770    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:46.975171 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:46.975123    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"]
May 28 19:48:47.068457 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068420    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-local-certs\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.068665 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068506    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.068665 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068531    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.068665 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068571    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l5v2x\" (UniqueName: \"kubernetes.io/projected/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-kube-api-access-l5v2x\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.068665 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068590    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-token\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.068665 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068615    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-cacerts\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.068665 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.068651    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.069222 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.069192    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.071090 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.071066    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-cacerts\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.071180 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.071122    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.071180 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.071127    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-local-certs\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.071579 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.071560    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.081095 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.081070    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-istio-token\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.081433 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.081414    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5v2x\" (UniqueName: \"kubernetes.io/projected/30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89-kube-api-access-l5v2x\") pod \"istiod-openshift-gateway-798958bb55-2fgcr\" (UID: \"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.263992 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.263887    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:47.433571 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.429362    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"]
May 28 19:48:47.435086 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:48:47.435053    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e3f39e_fbb7_4fdf_8cc3_e1800b32ff89.slice/crio-56294e30220b2d90f97f7239640c3516d0cbf574cbb0c2a761acf41ec1d34ce3 WatchSource:0}: Error finding container 56294e30220b2d90f97f7239640c3516d0cbf574cbb0c2a761acf41ec1d34ce3: Status 404 returned error can't find the container with id 56294e30220b2d90f97f7239640c3516d0cbf574cbb0c2a761acf41ec1d34ce3
May 28 19:48:47.680185 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:47.680145    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr" event={"ID":"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89","Type":"ContainerStarted","Data":"56294e30220b2d90f97f7239640c3516d0cbf574cbb0c2a761acf41ec1d34ce3"}
May 28 19:48:49.967099 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:49.967031    2568 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 28 19:48:49.967441 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:49.967140    2568 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
May 28 19:48:50.692650 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:50.692607    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr" event={"ID":"30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89","Type":"ContainerStarted","Data":"86868465108e3a98fce70fadb9d9a200a7283d1d3d5ba2aed595d95ec85f2e50"}
May 28 19:48:50.692824 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:50.692760    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:50.717028 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:50.716957    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr" podStartSLOduration=2.187599802 podStartE2EDuration="4.71693625s" podCreationTimestamp="2026-05-28 19:48:46 +0000 UTC" firstStartedPulling="2026-05-28 19:48:47.437394016 +0000 UTC m=+340.724734899" lastFinishedPulling="2026-05-28 19:48:49.966730453 +0000 UTC m=+343.254071347" observedRunningTime="2026-05-28 19:48:50.715441257 +0000 UTC m=+344.002782162" watchObservedRunningTime="2026-05-28 19:48:50.71693625 +0000 UTC m=+344.004277155"
May 28 19:48:51.698257 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:51.698229    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-2fgcr"
May 28 19:48:52.662547 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:52.662515    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-g7chn"
May 28 19:48:57.682240 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:48:57.682165    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-rjql7"
May 28 19:49:06.640133 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:06.640098    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-tfpxs"
May 28 19:49:59.097154 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.097112    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-kcmfw"]
May 28 19:49:59.100273 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.100255    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:49:59.102862 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.102827    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 28 19:49:59.103010 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.102836    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 28 19:49:59.103825 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.103806    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vgsbt\""
May 28 19:49:59.108934 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.108902    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-kcmfw"]
May 28 19:49:59.179669 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.179617    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zml\" (UniqueName: \"kubernetes.io/projected/0f65f48f-e9b2-4495-bcf4-efe039dc953a-kube-api-access-z9zml\") pod \"authorino-operator-657f44b778-kcmfw\" (UID: \"0f65f48f-e9b2-4495-bcf4-efe039dc953a\") " pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:49:59.280854 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.280806    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zml\" (UniqueName: \"kubernetes.io/projected/0f65f48f-e9b2-4495-bcf4-efe039dc953a-kube-api-access-z9zml\") pod \"authorino-operator-657f44b778-kcmfw\" (UID: \"0f65f48f-e9b2-4495-bcf4-efe039dc953a\") " pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:49:59.289996 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.289966    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zml\" (UniqueName: \"kubernetes.io/projected/0f65f48f-e9b2-4495-bcf4-efe039dc953a-kube-api-access-z9zml\") pod \"authorino-operator-657f44b778-kcmfw\" (UID: \"0f65f48f-e9b2-4495-bcf4-efe039dc953a\") " pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:49:59.411804 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.411716    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:49:59.549383 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.549356    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-kcmfw"]
May 28 19:49:59.552375 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:49:59.552345    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f65f48f_e9b2_4495_bcf4_efe039dc953a.slice/crio-8ab2b15e92dd818c47707368a743260e43fd997c79c1e8511f993f02d470437c WatchSource:0}: Error finding container 8ab2b15e92dd818c47707368a743260e43fd997c79c1e8511f993f02d470437c: Status 404 returned error can't find the container with id 8ab2b15e92dd818c47707368a743260e43fd997c79c1e8511f993f02d470437c
May 28 19:49:59.926657 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:49:59.926619    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw" event={"ID":"0f65f48f-e9b2-4495-bcf4-efe039dc953a","Type":"ContainerStarted","Data":"8ab2b15e92dd818c47707368a743260e43fd997c79c1e8511f993f02d470437c"}
May 28 19:50:01.935001 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:01.934963    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw" event={"ID":"0f65f48f-e9b2-4495-bcf4-efe039dc953a","Type":"ContainerStarted","Data":"ff46840c9ad0d9b67bcaa436a3cf5b4bf8d7aa300c4f6daaac5c62e60f301943"}
May 28 19:50:01.935491 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:01.935064    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:50:12.941187 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:12.941151    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw"
May 28 19:50:12.970111 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:12.970045    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-kcmfw" podStartSLOduration=12.34152947 podStartE2EDuration="13.970029943s" podCreationTimestamp="2026-05-28 19:49:59 +0000 UTC" firstStartedPulling="2026-05-28 19:49:59.554239026 +0000 UTC m=+412.841579907" lastFinishedPulling="2026-05-28 19:50:01.182739498 +0000 UTC m=+414.470080380" observedRunningTime="2026-05-28 19:50:01.953228754 +0000 UTC m=+415.240569671" watchObservedRunningTime="2026-05-28 19:50:12.970029943 +0000 UTC m=+426.257370847"
May 28 19:50:56.805293 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.805250    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:50:56.808633 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.808609    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:56.811452 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.811426    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 28 19:50:56.811627 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.811549    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gh5fm\""
May 28 19:50:56.819537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.819506    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:50:56.873683 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.873646    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsp64\" (UniqueName: \"kubernetes.io/projected/f36e05d3-7f6b-4c2c-a873-6bd62a457754-kube-api-access-qsp64\") pod \"limitador-limitador-7d549b5b-jw6x9\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") " pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:56.873877 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.873756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f36e05d3-7f6b-4c2c-a873-6bd62a457754-config-file\") pod \"limitador-limitador-7d549b5b-jw6x9\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") " pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:56.900986 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.900949    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:50:56.974708 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.974670    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f36e05d3-7f6b-4c2c-a873-6bd62a457754-config-file\") pod \"limitador-limitador-7d549b5b-jw6x9\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") " pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:56.974894 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.974721    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qsp64\" (UniqueName: \"kubernetes.io/projected/f36e05d3-7f6b-4c2c-a873-6bd62a457754-kube-api-access-qsp64\") pod \"limitador-limitador-7d549b5b-jw6x9\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") " pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:56.975292 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.975274    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f36e05d3-7f6b-4c2c-a873-6bd62a457754-config-file\") pod \"limitador-limitador-7d549b5b-jw6x9\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") " pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:56.985891 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:56.985862    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsp64\" (UniqueName: \"kubernetes.io/projected/f36e05d3-7f6b-4c2c-a873-6bd62a457754-kube-api-access-qsp64\") pod \"limitador-limitador-7d549b5b-jw6x9\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") " pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:57.120421 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:57.120333    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:50:57.315748 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:57.315718    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:50:57.317799 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:50:57.317762    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36e05d3_7f6b_4c2c_a873_6bd62a457754.slice/crio-bea9def6ed1d09fa5743f1f1bfcbb9cdae186f80f3336007525a189e1784b97a WatchSource:0}: Error finding container bea9def6ed1d09fa5743f1f1bfcbb9cdae186f80f3336007525a189e1784b97a: Status 404 returned error can't find the container with id bea9def6ed1d09fa5743f1f1bfcbb9cdae186f80f3336007525a189e1784b97a
May 28 19:50:58.125815 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:50:58.125758    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9" event={"ID":"f36e05d3-7f6b-4c2c-a873-6bd62a457754","Type":"ContainerStarted","Data":"bea9def6ed1d09fa5743f1f1bfcbb9cdae186f80f3336007525a189e1784b97a"}
May 28 19:51:00.135181 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:00.135139    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9" event={"ID":"f36e05d3-7f6b-4c2c-a873-6bd62a457754","Type":"ContainerStarted","Data":"020eb70a3b2a52007adfd5d95762eaad08e3e2865cf94803bb6101cd953215c2"}
May 28 19:51:00.135582 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:00.135205    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:51:00.153421 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:00.153370    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9" podStartSLOduration=1.47698409 podStartE2EDuration="4.153355407s" podCreationTimestamp="2026-05-28 19:50:56 +0000 UTC" firstStartedPulling="2026-05-28 19:50:57.319857409 +0000 UTC m=+470.607198295" lastFinishedPulling="2026-05-28 19:50:59.996228728 +0000 UTC m=+473.283569612" observedRunningTime="2026-05-28 19:51:00.153119397 +0000 UTC m=+473.440460302" watchObservedRunningTime="2026-05-28 19:51:00.153355407 +0000 UTC m=+473.440696310"
May 28 19:51:11.140620 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:11.140582    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:51:11.737630 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:11.737592    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:51:11.737836 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:11.737812    2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9" podUID="f36e05d3-7f6b-4c2c-a873-6bd62a457754" containerName="limitador" containerID="cri-o://020eb70a3b2a52007adfd5d95762eaad08e3e2865cf94803bb6101cd953215c2" gracePeriod=30
May 28 19:51:12.187080 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.187044    2568 generic.go:358] "Generic (PLEG): container finished" podID="f36e05d3-7f6b-4c2c-a873-6bd62a457754" containerID="020eb70a3b2a52007adfd5d95762eaad08e3e2865cf94803bb6101cd953215c2" exitCode=0
May 28 19:51:12.187434 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.187125    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9" event={"ID":"f36e05d3-7f6b-4c2c-a873-6bd62a457754","Type":"ContainerDied","Data":"020eb70a3b2a52007adfd5d95762eaad08e3e2865cf94803bb6101cd953215c2"}
May 28 19:51:12.683805 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.683779    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:51:12.814306 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.814217    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsp64\" (UniqueName: \"kubernetes.io/projected/f36e05d3-7f6b-4c2c-a873-6bd62a457754-kube-api-access-qsp64\") pod \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") "
May 28 19:51:12.814306 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.814250    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f36e05d3-7f6b-4c2c-a873-6bd62a457754-config-file\") pod \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\" (UID: \"f36e05d3-7f6b-4c2c-a873-6bd62a457754\") "
May 28 19:51:12.814675 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.814650    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36e05d3-7f6b-4c2c-a873-6bd62a457754-config-file" (OuterVolumeSpecName: "config-file") pod "f36e05d3-7f6b-4c2c-a873-6bd62a457754" (UID: "f36e05d3-7f6b-4c2c-a873-6bd62a457754"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 19:51:12.816679 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.816650    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36e05d3-7f6b-4c2c-a873-6bd62a457754-kube-api-access-qsp64" (OuterVolumeSpecName: "kube-api-access-qsp64") pod "f36e05d3-7f6b-4c2c-a873-6bd62a457754" (UID: "f36e05d3-7f6b-4c2c-a873-6bd62a457754"). InnerVolumeSpecName "kube-api-access-qsp64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 19:51:12.915047 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.915007    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsp64\" (UniqueName: \"kubernetes.io/projected/f36e05d3-7f6b-4c2c-a873-6bd62a457754-kube-api-access-qsp64\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:51:12.915047 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:12.915041    2568 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f36e05d3-7f6b-4c2c-a873-6bd62a457754-config-file\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:51:13.192304 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:13.192222    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9"
May 28 19:51:13.192304 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:13.192234    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-jw6x9" event={"ID":"f36e05d3-7f6b-4c2c-a873-6bd62a457754","Type":"ContainerDied","Data":"bea9def6ed1d09fa5743f1f1bfcbb9cdae186f80f3336007525a189e1784b97a"}
May 28 19:51:13.192304 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:13.192285    2568 scope.go:117] "RemoveContainer" containerID="020eb70a3b2a52007adfd5d95762eaad08e3e2865cf94803bb6101cd953215c2"
May 28 19:51:13.214564 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:13.214520    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:51:13.217973 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:13.217943    2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-jw6x9"]
May 28 19:51:13.333857 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:13.333819    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36e05d3-7f6b-4c2c-a873-6bd62a457754" path="/var/lib/kubelet/pods/f36e05d3-7f6b-4c2c-a873-6bd62a457754/volumes"
May 28 19:51:14.033045 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.033006    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-fldds"]
May 28 19:51:14.033352 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.033338    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36e05d3-7f6b-4c2c-a873-6bd62a457754" containerName="limitador"
May 28 19:51:14.033352 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.033353    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36e05d3-7f6b-4c2c-a873-6bd62a457754" containerName="limitador"
May 28 19:51:14.033501 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.033439    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="f36e05d3-7f6b-4c2c-a873-6bd62a457754" containerName="limitador"
May 28 19:51:14.037938 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.037909    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.041656 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.041624    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kpc8b\""
May 28 19:51:14.041801 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.041622    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 28 19:51:14.048712 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.048677    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-fldds"]
May 28 19:51:14.126787 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.126748    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b41c230d-5c52-4ecd-a0e2-a9487b7dd896-data\") pod \"postgres-68c4fbbd6f-fldds\" (UID: \"b41c230d-5c52-4ecd-a0e2-a9487b7dd896\") " pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.126972 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.126799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hg6m\" (UniqueName: \"kubernetes.io/projected/b41c230d-5c52-4ecd-a0e2-a9487b7dd896-kube-api-access-9hg6m\") pod \"postgres-68c4fbbd6f-fldds\" (UID: \"b41c230d-5c52-4ecd-a0e2-a9487b7dd896\") " pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.227824 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.227780    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b41c230d-5c52-4ecd-a0e2-a9487b7dd896-data\") pod \"postgres-68c4fbbd6f-fldds\" (UID: \"b41c230d-5c52-4ecd-a0e2-a9487b7dd896\") " pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.227824 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.227824    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hg6m\" (UniqueName: \"kubernetes.io/projected/b41c230d-5c52-4ecd-a0e2-a9487b7dd896-kube-api-access-9hg6m\") pod \"postgres-68c4fbbd6f-fldds\" (UID: \"b41c230d-5c52-4ecd-a0e2-a9487b7dd896\") " pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.228230 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.228216    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b41c230d-5c52-4ecd-a0e2-a9487b7dd896-data\") pod \"postgres-68c4fbbd6f-fldds\" (UID: \"b41c230d-5c52-4ecd-a0e2-a9487b7dd896\") " pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.241160 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.241133    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hg6m\" (UniqueName: \"kubernetes.io/projected/b41c230d-5c52-4ecd-a0e2-a9487b7dd896-kube-api-access-9hg6m\") pod \"postgres-68c4fbbd6f-fldds\" (UID: \"b41c230d-5c52-4ecd-a0e2-a9487b7dd896\") " pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.350956 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.350897    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:14.494613 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:14.494593    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-fldds"]
May 28 19:51:14.497379 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:51:14.497341    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb41c230d_5c52_4ecd_a0e2_a9487b7dd896.slice/crio-f317d2674d70aecd6719823633aba6074526dec7f7f579307120cbe88dcbf61a WatchSource:0}: Error finding container f317d2674d70aecd6719823633aba6074526dec7f7f579307120cbe88dcbf61a: Status 404 returned error can't find the container with id f317d2674d70aecd6719823633aba6074526dec7f7f579307120cbe88dcbf61a
May 28 19:51:15.204537 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:15.204445    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-fldds" event={"ID":"b41c230d-5c52-4ecd-a0e2-a9487b7dd896","Type":"ContainerStarted","Data":"f317d2674d70aecd6719823633aba6074526dec7f7f579307120cbe88dcbf61a"}
May 28 19:51:20.224947 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:20.224909    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-fldds" event={"ID":"b41c230d-5c52-4ecd-a0e2-a9487b7dd896","Type":"ContainerStarted","Data":"f19eb00c046d5725c9142d2e2a900dab10131b06cb91e17458688a07ba13700b"}
May 28 19:51:20.225390 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:20.225027    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:20.245205 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:20.245148    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-fldds" podStartSLOduration=1.382551793 podStartE2EDuration="6.245133348s" podCreationTimestamp="2026-05-28 19:51:14 +0000 UTC" firstStartedPulling="2026-05-28 19:51:14.498736797 +0000 UTC m=+487.786077679" lastFinishedPulling="2026-05-28 19:51:19.361318349 +0000 UTC m=+492.648659234" observedRunningTime="2026-05-28 19:51:20.243549745 +0000 UTC m=+493.530890649" watchObservedRunningTime="2026-05-28 19:51:20.245133348 +0000 UTC m=+493.532474253"
May 28 19:51:26.261110 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:26.261071    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-fldds"
May 28 19:51:31.952420 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:31.952382    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f64df6bcf-dsmcx"]
May 28 19:51:31.978128 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:31.978034    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f64df6bcf-dsmcx"]
May 28 19:51:31.978283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:31.978182    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:31.980984 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:31.980960    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 28 19:51:31.981155 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:31.981073    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jbls5\""
May 28 19:51:32.095520 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:32.095444    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjnpt\" (UniqueName: \"kubernetes.io/projected/08b88a1a-b62a-49bd-88c0-1be6e67981e8-kube-api-access-zjnpt\") pod \"maas-controller-f64df6bcf-dsmcx\" (UID: \"08b88a1a-b62a-49bd-88c0-1be6e67981e8\") " pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:32.196340 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:32.196303    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zjnpt\" (UniqueName: \"kubernetes.io/projected/08b88a1a-b62a-49bd-88c0-1be6e67981e8-kube-api-access-zjnpt\") pod \"maas-controller-f64df6bcf-dsmcx\" (UID: \"08b88a1a-b62a-49bd-88c0-1be6e67981e8\") " pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:32.205116 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:32.205038    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjnpt\" (UniqueName: \"kubernetes.io/projected/08b88a1a-b62a-49bd-88c0-1be6e67981e8-kube-api-access-zjnpt\") pod \"maas-controller-f64df6bcf-dsmcx\" (UID: \"08b88a1a-b62a-49bd-88c0-1be6e67981e8\") " pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:32.289208 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:32.289172    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:32.432598 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:32.432568    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f64df6bcf-dsmcx"]
May 28 19:51:32.435328 ip-10-0-135-191 kubenswrapper[2568]: W0528 19:51:32.435296    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b88a1a_b62a_49bd_88c0_1be6e67981e8.slice/crio-f47dfa82551ee098367e2ad7492be8019312ab4af882e684aa3a5362408fb5e4 WatchSource:0}: Error finding container f47dfa82551ee098367e2ad7492be8019312ab4af882e684aa3a5362408fb5e4: Status 404 returned error can't find the container with id f47dfa82551ee098367e2ad7492be8019312ab4af882e684aa3a5362408fb5e4
May 28 19:51:33.273617 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:33.273574    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f64df6bcf-dsmcx" event={"ID":"08b88a1a-b62a-49bd-88c0-1be6e67981e8","Type":"ContainerStarted","Data":"f47dfa82551ee098367e2ad7492be8019312ab4af882e684aa3a5362408fb5e4"}
May 28 19:51:35.282848 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:35.282803    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f64df6bcf-dsmcx" event={"ID":"08b88a1a-b62a-49bd-88c0-1be6e67981e8","Type":"ContainerStarted","Data":"420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1"}
May 28 19:51:35.283260 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:35.282935    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:35.312283 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:35.312233    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-f64df6bcf-dsmcx" podStartSLOduration=1.9509731860000001 podStartE2EDuration="4.3122175s" podCreationTimestamp="2026-05-28 19:51:31 +0000 UTC" firstStartedPulling="2026-05-28 19:51:32.436607352 +0000 UTC m=+505.723948234" lastFinishedPulling="2026-05-28 19:51:34.797851651 +0000 UTC m=+508.085192548" observedRunningTime="2026-05-28 19:51:35.311763407 +0000 UTC m=+508.599104313" watchObservedRunningTime="2026-05-28 19:51:35.3122175 +0000 UTC m=+508.599558404"
May 28 19:51:45.892786 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:45.892745    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f64df6bcf-dsmcx"]
May 28 19:51:45.893292 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:45.892976    2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-f64df6bcf-dsmcx" podUID="08b88a1a-b62a-49bd-88c0-1be6e67981e8" containerName="manager" containerID="cri-o://420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1" gracePeriod=30
May 28 19:51:45.900446 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:45.900419    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:46.139139 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.139113    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:46.223866 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.223822    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjnpt\" (UniqueName: \"kubernetes.io/projected/08b88a1a-b62a-49bd-88c0-1be6e67981e8-kube-api-access-zjnpt\") pod \"08b88a1a-b62a-49bd-88c0-1be6e67981e8\" (UID: \"08b88a1a-b62a-49bd-88c0-1be6e67981e8\") "
May 28 19:51:46.226136 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.226109    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b88a1a-b62a-49bd-88c0-1be6e67981e8-kube-api-access-zjnpt" (OuterVolumeSpecName: "kube-api-access-zjnpt") pod "08b88a1a-b62a-49bd-88c0-1be6e67981e8" (UID: "08b88a1a-b62a-49bd-88c0-1be6e67981e8"). InnerVolumeSpecName "kube-api-access-zjnpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 19:51:46.324581 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.324547    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjnpt\" (UniqueName: \"kubernetes.io/projected/08b88a1a-b62a-49bd-88c0-1be6e67981e8-kube-api-access-zjnpt\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 19:51:46.330389 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.330359    2568 generic.go:358] "Generic (PLEG): container finished" podID="08b88a1a-b62a-49bd-88c0-1be6e67981e8" containerID="420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1" exitCode=0
May 28 19:51:46.330572 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.330394    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f64df6bcf-dsmcx" event={"ID":"08b88a1a-b62a-49bd-88c0-1be6e67981e8","Type":"ContainerDied","Data":"420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1"}
May 28 19:51:46.330572 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.330406    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f64df6bcf-dsmcx"
May 28 19:51:46.330572 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.330415    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f64df6bcf-dsmcx" event={"ID":"08b88a1a-b62a-49bd-88c0-1be6e67981e8","Type":"ContainerDied","Data":"f47dfa82551ee098367e2ad7492be8019312ab4af882e684aa3a5362408fb5e4"}
May 28 19:51:46.330572 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.330430    2568 scope.go:117] "RemoveContainer" containerID="420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1"
May 28 19:51:46.339066 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.339048    2568 scope.go:117] "RemoveContainer" containerID="420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1"
May 28 19:51:46.339369 ip-10-0-135-191 kubenswrapper[2568]: E0528 19:51:46.339348    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1\": container with ID starting with 420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1 not found: ID does not exist" containerID="420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1"
May 28 19:51:46.339417 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.339378    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1"} err="failed to get container status \"420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1\": rpc error: code = NotFound desc = could not find container \"420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1\": container with ID starting with 420b5eac68a271ac57546860e08a98a6a99d2584d94fcc15a11a7abeb92035f1 not found: ID does not exist"
May 28 19:51:46.353035 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.353000    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f64df6bcf-dsmcx"]
May 28 19:51:46.357208 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:46.357168    2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f64df6bcf-dsmcx"]
May 28 19:51:47.337398 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:51:47.336070    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08b88a1a-b62a-49bd-88c0-1be6e67981e8" path="/var/lib/kubelet/pods/08b88a1a-b62a-49bd-88c0-1be6e67981e8/volumes"
May 28 19:53:07.260766 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:53:07.260735    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:53:07.263093 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:53:07.263066    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:53:07.265344 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:53:07.265323    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:53:07.267582 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:53:07.267559    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:58:07.282860 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:58:07.282771    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:58:07.287372 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:58:07.287349    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 19:58:07.288020 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:58:07.288001    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 19:58:07.292633 ip-10-0-135-191 kubenswrapper[2568]: I0528 19:58:07.292486    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:00:00.149183 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.149144    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29666640-bldz2"]
May 28 20:00:00.151495 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.149523    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b88a1a-b62a-49bd-88c0-1be6e67981e8" containerName="manager"
May 28 20:00:00.151495 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.149535    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b88a1a-b62a-49bd-88c0-1be6e67981e8" containerName="manager"
May 28 20:00:00.151495 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.149612    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="08b88a1a-b62a-49bd-88c0-1be6e67981e8" containerName="manager"
May 28 20:00:00.152505 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.152489    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:00.155320 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.155299    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mvrvv\""
May 28 20:00:00.161008 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.160984    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666640-bldz2"]
May 28 20:00:00.278048 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.278012    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svpz4\" (UniqueName: \"kubernetes.io/projected/9e22dde4-322e-44a6-9e79-21c289dad73a-kube-api-access-svpz4\") pod \"maas-api-key-cleanup-29666640-bldz2\" (UID: \"9e22dde4-322e-44a6-9e79-21c289dad73a\") " pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:00.379395 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.379350    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-svpz4\" (UniqueName: \"kubernetes.io/projected/9e22dde4-322e-44a6-9e79-21c289dad73a-kube-api-access-svpz4\") pod \"maas-api-key-cleanup-29666640-bldz2\" (UID: \"9e22dde4-322e-44a6-9e79-21c289dad73a\") " pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:00.387935 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.387907    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svpz4\" (UniqueName: \"kubernetes.io/projected/9e22dde4-322e-44a6-9e79-21c289dad73a-kube-api-access-svpz4\") pod \"maas-api-key-cleanup-29666640-bldz2\" (UID: \"9e22dde4-322e-44a6-9e79-21c289dad73a\") " pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:00.464329 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.464241    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:00.603371 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.603336    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666640-bldz2"]
May 28 20:00:00.606584 ip-10-0-135-191 kubenswrapper[2568]: W0528 20:00:00.606545    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e22dde4_322e_44a6_9e79_21c289dad73a.slice/crio-565c3f432cd9b9f8ba9292adac57c88e1af9fb5f26c7930def39c10e801b92a5 WatchSource:0}: Error finding container 565c3f432cd9b9f8ba9292adac57c88e1af9fb5f26c7930def39c10e801b92a5: Status 404 returned error can't find the container with id 565c3f432cd9b9f8ba9292adac57c88e1af9fb5f26c7930def39c10e801b92a5
May 28 20:00:00.608391 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:00.608372    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 20:00:01.017258 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:01.017222    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" event={"ID":"9e22dde4-322e-44a6-9e79-21c289dad73a","Type":"ContainerStarted","Data":"565c3f432cd9b9f8ba9292adac57c88e1af9fb5f26c7930def39c10e801b92a5"}
May 28 20:00:06.037903 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:06.037865    2568 generic.go:358] "Generic (PLEG): container finished" podID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerID="c661737ab9190ea157247248e118b17f1159c1309f7bbfeff5239fda51e45536" exitCode=7
May 28 20:00:06.038272 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:06.037966    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" event={"ID":"9e22dde4-322e-44a6-9e79-21c289dad73a","Type":"ContainerDied","Data":"c661737ab9190ea157247248e118b17f1159c1309f7bbfeff5239fda51e45536"}
May 28 20:00:06.038272 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:06.038179    2568 scope.go:117] "RemoveContainer" containerID="c661737ab9190ea157247248e118b17f1159c1309f7bbfeff5239fda51e45536"
May 28 20:00:07.043833 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:07.043739    2568 generic.go:358] "Generic (PLEG): container finished" podID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerID="22add53513c6eb3602f10dc58525ac4d6ddcfd6118965288e6ec4d8fe2b46563" exitCode=7
May 28 20:00:07.043833 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:07.043820    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" event={"ID":"9e22dde4-322e-44a6-9e79-21c289dad73a","Type":"ContainerDied","Data":"22add53513c6eb3602f10dc58525ac4d6ddcfd6118965288e6ec4d8fe2b46563"}
May 28 20:00:07.044293 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:07.043867    2568 scope.go:117] "RemoveContainer" containerID="c661737ab9190ea157247248e118b17f1159c1309f7bbfeff5239fda51e45536"
May 28 20:00:07.044293 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:07.044034    2568 scope.go:117] "RemoveContainer" containerID="22add53513c6eb3602f10dc58525ac4d6ddcfd6118965288e6ec4d8fe2b46563"
May 28 20:00:07.044293 ip-10-0-135-191 kubenswrapper[2568]: E0528 20:00:07.044263    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29666640-bldz2_opendatahub(9e22dde4-322e-44a6-9e79-21c289dad73a)\"" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a"
May 28 20:00:08.048715 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:08.048688    2568 scope.go:117] "RemoveContainer" containerID="22add53513c6eb3602f10dc58525ac4d6ddcfd6118965288e6ec4d8fe2b46563"
May 28 20:00:08.049090 ip-10-0-135-191 kubenswrapper[2568]: E0528 20:00:08.048878    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29666640-bldz2_opendatahub(9e22dde4-322e-44a6-9e79-21c289dad73a)\"" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a"
May 28 20:00:22.330409 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:22.330373    2568 scope.go:117] "RemoveContainer" containerID="22add53513c6eb3602f10dc58525ac4d6ddcfd6118965288e6ec4d8fe2b46563"
May 28 20:00:23.101722 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:23.101686    2568 generic.go:358] "Generic (PLEG): container finished" podID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerID="5c6d287a6a41cc86b84312019bb62c1834672ab415a12c33841f21e95377d443" exitCode=7
May 28 20:00:23.101919 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:23.101767    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" event={"ID":"9e22dde4-322e-44a6-9e79-21c289dad73a","Type":"ContainerDied","Data":"5c6d287a6a41cc86b84312019bb62c1834672ab415a12c33841f21e95377d443"}
May 28 20:00:23.101919 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:23.101820    2568 scope.go:117] "RemoveContainer" containerID="22add53513c6eb3602f10dc58525ac4d6ddcfd6118965288e6ec4d8fe2b46563"
May 28 20:00:23.102230 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:23.102209    2568 scope.go:117] "RemoveContainer" containerID="5c6d287a6a41cc86b84312019bb62c1834672ab415a12c33841f21e95377d443"
May 28 20:00:23.102518 ip-10-0-135-191 kubenswrapper[2568]: E0528 20:00:23.102460    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29666640-bldz2_opendatahub(9e22dde4-322e-44a6-9e79-21c289dad73a)\"" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a"
May 28 20:00:24.130662 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:24.130626    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666640-bldz2"]
May 28 20:00:24.268651 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:24.268616    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:24.389245 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:24.389140    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svpz4\" (UniqueName: \"kubernetes.io/projected/9e22dde4-322e-44a6-9e79-21c289dad73a-kube-api-access-svpz4\") pod \"9e22dde4-322e-44a6-9e79-21c289dad73a\" (UID: \"9e22dde4-322e-44a6-9e79-21c289dad73a\") "
May 28 20:00:24.391533 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:24.391500    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e22dde4-322e-44a6-9e79-21c289dad73a-kube-api-access-svpz4" (OuterVolumeSpecName: "kube-api-access-svpz4") pod "9e22dde4-322e-44a6-9e79-21c289dad73a" (UID: "9e22dde4-322e-44a6-9e79-21c289dad73a"). InnerVolumeSpecName "kube-api-access-svpz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 20:00:24.489779 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:24.489748    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svpz4\" (UniqueName: \"kubernetes.io/projected/9e22dde4-322e-44a6-9e79-21c289dad73a-kube-api-access-svpz4\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 20:00:25.110592 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:25.110551    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2" event={"ID":"9e22dde4-322e-44a6-9e79-21c289dad73a","Type":"ContainerDied","Data":"565c3f432cd9b9f8ba9292adac57c88e1af9fb5f26c7930def39c10e801b92a5"}
May 28 20:00:25.110592 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:25.110581    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666640-bldz2"
May 28 20:00:25.110592 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:25.110595    2568 scope.go:117] "RemoveContainer" containerID="5c6d287a6a41cc86b84312019bb62c1834672ab415a12c33841f21e95377d443"
May 28 20:00:25.133249 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:25.133215    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666640-bldz2"]
May 28 20:00:25.136627 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:25.136599    2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666640-bldz2"]
May 28 20:00:25.337163 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:00:25.337128    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" path="/var/lib/kubelet/pods/9e22dde4-322e-44a6-9e79-21c289dad73a/volumes"
May 28 20:03:07.304456 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:03:07.304426    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:03:07.308787 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:03:07.308765    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:03:07.311431 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:03:07.311408    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:03:07.315254 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:03:07.315239    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:08:07.327195 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:08:07.327161    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:08:07.332167 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:08:07.332146    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:08:07.341754 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:08:07.341721    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:08:07.345903 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:08:07.345879    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:12:07.654773 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.654733    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"]
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655094    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655107    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655121    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655129    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655193    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655206    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655269    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655275    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.657187 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.655330    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="9e22dde4-322e-44a6-9e79-21c289dad73a" containerName="cleanup"
May 28 20:12:07.658079 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.658064    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.662051 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.662016    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 28 20:12:07.662190 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.662049    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 28 20:12:07.662190 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.662016    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 28 20:12:07.662190 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.662025    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vqc2b\""
May 28 20:12:07.681453 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.681419    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"]
May 28 20:12:07.808080 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.808042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr2dk\" (UniqueName: \"kubernetes.io/projected/dd87511a-849c-4ab3-8fff-ade9399f0e15-kube-api-access-cr2dk\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.808265 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.808091    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.808265 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.808113    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.808265 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.808185    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.808265 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.808211    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.808265 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.808233    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd87511a-849c-4ab3-8fff-ade9399f0e15-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909057 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.908959    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cr2dk\" (UniqueName: \"kubernetes.io/projected/dd87511a-849c-4ab3-8fff-ade9399f0e15-kube-api-access-cr2dk\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909057 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909028    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909285 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909061    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909285 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909098    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909285 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909127    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909285 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909161    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd87511a-849c-4ab3-8fff-ade9399f0e15-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909536 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909509    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909738 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909541    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.909848 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.909771    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.911794 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.911766    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd87511a-849c-4ab3-8fff-ade9399f0e15-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.912168 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.912138    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd87511a-849c-4ab3-8fff-ade9399f0e15-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.918585 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.918554    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr2dk\" (UniqueName: \"kubernetes.io/projected/dd87511a-849c-4ab3-8fff-ade9399f0e15-kube-api-access-cr2dk\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9\" (UID: \"dd87511a-849c-4ab3-8fff-ade9399f0e15\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:07.968841 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:07.968803    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:08.155774 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:08.155738    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"]
May 28 20:12:08.159501 ip-10-0-135-191 kubenswrapper[2568]: W0528 20:12:08.159414    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd87511a_849c_4ab3_8fff_ade9399f0e15.slice/crio-675fe030c0d19954c670d3aabc92bdd92fcbb08c694de1e009d7058d0685216e WatchSource:0}: Error finding container 675fe030c0d19954c670d3aabc92bdd92fcbb08c694de1e009d7058d0685216e: Status 404 returned error can't find the container with id 675fe030c0d19954c670d3aabc92bdd92fcbb08c694de1e009d7058d0685216e
May 28 20:12:08.161186 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:08.161167    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 20:12:08.531208 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:08.531111    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9" event={"ID":"dd87511a-849c-4ab3-8fff-ade9399f0e15","Type":"ContainerStarted","Data":"675fe030c0d19954c670d3aabc92bdd92fcbb08c694de1e009d7058d0685216e"}
May 28 20:12:16.565659 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:16.565618    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9" event={"ID":"dd87511a-849c-4ab3-8fff-ade9399f0e15","Type":"ContainerStarted","Data":"64e19bc9238534ad05098c37a231802357bc09d2838657315a60c734153ce2aa"}
May 28 20:12:18.947629 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:18.947583    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"]
May 28 20:12:18.953003 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:18.952979    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:18.956096 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:18.956072    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
May 28 20:12:18.963792 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:18.963761    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"]
May 28 20:12:19.002010 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.001960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.002210 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.002026    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.002210 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.002057    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.002210 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.002090    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.002210 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.002132    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f812fb9c-5011-4ca0-9b29-cef2e22672e6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.002210 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.002179    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhp78\" (UniqueName: \"kubernetes.io/projected/f812fb9c-5011-4ca0-9b29-cef2e22672e6-kube-api-access-fhp78\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.103738 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.103673    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f812fb9c-5011-4ca0-9b29-cef2e22672e6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.104379 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.104351    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fhp78\" (UniqueName: \"kubernetes.io/projected/f812fb9c-5011-4ca0-9b29-cef2e22672e6-kube-api-access-fhp78\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.105049 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.105031    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.105605 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.105583    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.105728 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.105620    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.105728 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.105660    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.106064 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.106043    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.106369 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.106347    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.106569 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.106544    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.108032 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.107997    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f812fb9c-5011-4ca0-9b29-cef2e22672e6-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.108375 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.108352    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f812fb9c-5011-4ca0-9b29-cef2e22672e6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.113932 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.113899    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhp78\" (UniqueName: \"kubernetes.io/projected/f812fb9c-5011-4ca0-9b29-cef2e22672e6-kube-api-access-fhp78\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl\" (UID: \"f812fb9c-5011-4ca0-9b29-cef2e22672e6\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.265316 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.265227    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:19.423749 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.423717    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"]
May 28 20:12:19.426619 ip-10-0-135-191 kubenswrapper[2568]: W0528 20:12:19.426589    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf812fb9c_5011_4ca0_9b29_cef2e22672e6.slice/crio-c4486e274c62a6ca84f19e32fd331233c44734d60e457b58c406b5518a2ae257 WatchSource:0}: Error finding container c4486e274c62a6ca84f19e32fd331233c44734d60e457b58c406b5518a2ae257: Status 404 returned error can't find the container with id c4486e274c62a6ca84f19e32fd331233c44734d60e457b58c406b5518a2ae257
May 28 20:12:19.578081 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.578044    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl" event={"ID":"f812fb9c-5011-4ca0-9b29-cef2e22672e6","Type":"ContainerStarted","Data":"5d1e8bc72057e62e79133e84eccefedd61fb8015e4494428a6a8dd4e41aa8b4b"}
May 28 20:12:19.578081 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:19.578085    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl" event={"ID":"f812fb9c-5011-4ca0-9b29-cef2e22672e6","Type":"ContainerStarted","Data":"c4486e274c62a6ca84f19e32fd331233c44734d60e457b58c406b5518a2ae257"}
May 28 20:12:21.149440 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.149396    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"]
May 28 20:12:21.154147 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.154120    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.157123 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.157100    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
May 28 20:12:21.166297 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.166267    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"]
May 28 20:12:21.228284 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.228242    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.228496 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.228308    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlkcc\" (UniqueName: \"kubernetes.io/projected/6c0699f6-268c-4e22-8a6a-9ef6372bc287-kube-api-access-vlkcc\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.228496 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.228373    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.228496 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.228402    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.228691 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.228563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0699f6-268c-4e22-8a6a-9ef6372bc287-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.228691 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.228616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.329812 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.329779    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0699f6-268c-4e22-8a6a-9ef6372bc287-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.329812 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.329827    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330044 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.329856    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330044 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.329882    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vlkcc\" (UniqueName: \"kubernetes.io/projected/6c0699f6-268c-4e22-8a6a-9ef6372bc287-kube-api-access-vlkcc\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330044 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.329931    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330044 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.329956    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330296 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.330272    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330390 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.330354    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.330687 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.330662    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.333274 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.333221    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c0699f6-268c-4e22-8a6a-9ef6372bc287-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.333397 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.333295    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c0699f6-268c-4e22-8a6a-9ef6372bc287-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.348934 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.348894    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlkcc\" (UniqueName: \"kubernetes.io/projected/6c0699f6-268c-4e22-8a6a-9ef6372bc287-kube-api-access-vlkcc\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-jnbct\" (UID: \"6c0699f6-268c-4e22-8a6a-9ef6372bc287\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.466015 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.465447    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:21.647931 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:21.647898    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"]
May 28 20:12:21.650075 ip-10-0-135-191 kubenswrapper[2568]: W0528 20:12:21.650040    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0699f6_268c_4e22_8a6a_9ef6372bc287.slice/crio-87bd36476dbd2e3868932c8cc4502a6e00d9a77c6f145defcdb1fbc8457a10f8 WatchSource:0}: Error finding container 87bd36476dbd2e3868932c8cc4502a6e00d9a77c6f145defcdb1fbc8457a10f8: Status 404 returned error can't find the container with id 87bd36476dbd2e3868932c8cc4502a6e00d9a77c6f145defcdb1fbc8457a10f8
May 28 20:12:22.594011 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:22.593916    2568 generic.go:358] "Generic (PLEG): container finished" podID="dd87511a-849c-4ab3-8fff-ade9399f0e15" containerID="64e19bc9238534ad05098c37a231802357bc09d2838657315a60c734153ce2aa" exitCode=0
May 28 20:12:22.594417 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:22.594004    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9" event={"ID":"dd87511a-849c-4ab3-8fff-ade9399f0e15","Type":"ContainerDied","Data":"64e19bc9238534ad05098c37a231802357bc09d2838657315a60c734153ce2aa"}
May 28 20:12:22.595561 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:22.595445    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct" event={"ID":"6c0699f6-268c-4e22-8a6a-9ef6372bc287","Type":"ContainerStarted","Data":"62da36a9503df8ae985a1a6a3a670b58165155e1c6515cf0841452306e46e685"}
May 28 20:12:22.595561 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:22.595500    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct" event={"ID":"6c0699f6-268c-4e22-8a6a-9ef6372bc287","Type":"ContainerStarted","Data":"87bd36476dbd2e3868932c8cc4502a6e00d9a77c6f145defcdb1fbc8457a10f8"}
May 28 20:12:24.606876 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:24.606834    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9" event={"ID":"dd87511a-849c-4ab3-8fff-ade9399f0e15","Type":"ContainerStarted","Data":"746c745972db13e508d9b08f4dc9417f7dee07d228fec5c3fc7efeeb6d55eccd"}
May 28 20:12:24.607430 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:24.607365    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:24.642174 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:24.642110    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9" podStartSLOduration=1.7069854100000001 podStartE2EDuration="17.642096028s" podCreationTimestamp="2026-05-28 20:12:07 +0000 UTC" firstStartedPulling="2026-05-28 20:12:08.161327266 +0000 UTC m=+1741.448668151" lastFinishedPulling="2026-05-28 20:12:24.096437881 +0000 UTC m=+1757.383778769" observedRunningTime="2026-05-28 20:12:24.640524172 +0000 UTC m=+1757.927865108" watchObservedRunningTime="2026-05-28 20:12:24.642096028 +0000 UTC m=+1757.929436932"
May 28 20:12:25.612852 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:25.612818    2568 generic.go:358] "Generic (PLEG): container finished" podID="f812fb9c-5011-4ca0-9b29-cef2e22672e6" containerID="5d1e8bc72057e62e79133e84eccefedd61fb8015e4494428a6a8dd4e41aa8b4b" exitCode=0
May 28 20:12:25.613297 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:25.612886    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl" event={"ID":"f812fb9c-5011-4ca0-9b29-cef2e22672e6","Type":"ContainerDied","Data":"5d1e8bc72057e62e79133e84eccefedd61fb8015e4494428a6a8dd4e41aa8b4b"}
May 28 20:12:26.619292 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:26.619255    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl" event={"ID":"f812fb9c-5011-4ca0-9b29-cef2e22672e6","Type":"ContainerStarted","Data":"b538573445fe2809521ed1b39cfa0e36ba08b5f04c1029a1458fad3bcae5a1a3"}
May 28 20:12:26.619764 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:26.619519    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:26.653443 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:26.653387    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl" podStartSLOduration=8.151721949 podStartE2EDuration="8.653371358s" podCreationTimestamp="2026-05-28 20:12:18 +0000 UTC" firstStartedPulling="2026-05-28 20:12:25.613538456 +0000 UTC m=+1758.900879338" lastFinishedPulling="2026-05-28 20:12:26.115187854 +0000 UTC m=+1759.402528747" observedRunningTime="2026-05-28 20:12:26.650226464 +0000 UTC m=+1759.937567369" watchObservedRunningTime="2026-05-28 20:12:26.653371358 +0000 UTC m=+1759.940712261"
May 28 20:12:31.638513 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:31.638450    2568 generic.go:358] "Generic (PLEG): container finished" podID="6c0699f6-268c-4e22-8a6a-9ef6372bc287" containerID="62da36a9503df8ae985a1a6a3a670b58165155e1c6515cf0841452306e46e685" exitCode=0
May 28 20:12:31.638513 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:31.638516    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct" event={"ID":"6c0699f6-268c-4e22-8a6a-9ef6372bc287","Type":"ContainerDied","Data":"62da36a9503df8ae985a1a6a3a670b58165155e1c6515cf0841452306e46e685"}
May 28 20:12:32.643844 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:32.643799    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct" event={"ID":"6c0699f6-268c-4e22-8a6a-9ef6372bc287","Type":"ContainerStarted","Data":"01e0f719077f546e7c0877c27bccf47a58ddcdb0ef7a8dabe0cc703034dd273b"}
May 28 20:12:32.644271 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:32.644016    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:12:32.666317 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:32.666236    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct" podStartSLOduration=11.359914499 podStartE2EDuration="11.666217737s" podCreationTimestamp="2026-05-28 20:12:21 +0000 UTC" firstStartedPulling="2026-05-28 20:12:31.639194726 +0000 UTC m=+1764.926535610" lastFinishedPulling="2026-05-28 20:12:31.945497966 +0000 UTC m=+1765.232838848" observedRunningTime="2026-05-28 20:12:32.663823933 +0000 UTC m=+1765.951164837" watchObservedRunningTime="2026-05-28 20:12:32.666217737 +0000 UTC m=+1765.953558642"
May 28 20:12:35.626807 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:35.626774    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9"
May 28 20:12:37.639650 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:37.639618    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl"
May 28 20:12:43.660879 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:12:43.660843    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-jnbct"
May 28 20:13:07.364560 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:13:07.364530    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:13:07.369154 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:13:07.369117    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:13:07.370726 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:13:07.370684    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:13:07.375258 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:13:07.375238    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:15:00.137758 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.137718    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29666655-zdchs"]
May 28 20:15:00.141231 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.141206    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:00.143715 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.143687    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mvrvv\""
May 28 20:15:00.156183 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.156155    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666655-zdchs"]
May 28 20:15:00.262513 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.262449    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr526\" (UniqueName: \"kubernetes.io/projected/355533f6-8ffa-4295-b679-074b729f11ba-kube-api-access-qr526\") pod \"maas-api-key-cleanup-29666655-zdchs\" (UID: \"355533f6-8ffa-4295-b679-074b729f11ba\") " pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:00.363981 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.363938    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qr526\" (UniqueName: \"kubernetes.io/projected/355533f6-8ffa-4295-b679-074b729f11ba-kube-api-access-qr526\") pod \"maas-api-key-cleanup-29666655-zdchs\" (UID: \"355533f6-8ffa-4295-b679-074b729f11ba\") " pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:00.373689 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.373654    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr526\" (UniqueName: \"kubernetes.io/projected/355533f6-8ffa-4295-b679-074b729f11ba-kube-api-access-qr526\") pod \"maas-api-key-cleanup-29666655-zdchs\" (UID: \"355533f6-8ffa-4295-b679-074b729f11ba\") " pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:00.451678 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.451602    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:00.590893 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:00.590821    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666655-zdchs"]
May 28 20:15:00.593262 ip-10-0-135-191 kubenswrapper[2568]: W0528 20:15:00.593209    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355533f6_8ffa_4295_b679_074b729f11ba.slice/crio-8375630c5198eaef89d7cd550256582e3c4e7ff3242103bbc4802f6c5027e2ff WatchSource:0}: Error finding container 8375630c5198eaef89d7cd550256582e3c4e7ff3242103bbc4802f6c5027e2ff: Status 404 returned error can't find the container with id 8375630c5198eaef89d7cd550256582e3c4e7ff3242103bbc4802f6c5027e2ff
May 28 20:15:01.162932 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:01.162893    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" event={"ID":"355533f6-8ffa-4295-b679-074b729f11ba","Type":"ContainerStarted","Data":"8ab2d1d5671168845b425d1e892024f7e0f8cd2b0c0d8b7e03896481670d00ed"}
May 28 20:15:01.162932 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:01.162934    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" event={"ID":"355533f6-8ffa-4295-b679-074b729f11ba","Type":"ContainerStarted","Data":"8375630c5198eaef89d7cd550256582e3c4e7ff3242103bbc4802f6c5027e2ff"}
May 28 20:15:01.178451 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:01.178398    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" podStartSLOduration=1.1783832300000001 podStartE2EDuration="1.17838323s" podCreationTimestamp="2026-05-28 20:15:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:15:01.17785816 +0000 UTC m=+1914.465199065" watchObservedRunningTime="2026-05-28 20:15:01.17838323 +0000 UTC m=+1914.465724134"
May 28 20:15:02.167124 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:02.167092    2568 generic.go:358] "Generic (PLEG): container finished" podID="355533f6-8ffa-4295-b679-074b729f11ba" containerID="8ab2d1d5671168845b425d1e892024f7e0f8cd2b0c0d8b7e03896481670d00ed" exitCode=7
May 28 20:15:02.167546 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:02.167168    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" event={"ID":"355533f6-8ffa-4295-b679-074b729f11ba","Type":"ContainerDied","Data":"8ab2d1d5671168845b425d1e892024f7e0f8cd2b0c0d8b7e03896481670d00ed"}
May 28 20:15:02.167546 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:02.167328    2568 scope.go:117] "RemoveContainer" containerID="8ab2d1d5671168845b425d1e892024f7e0f8cd2b0c0d8b7e03896481670d00ed"
May 28 20:15:03.173121 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:03.173013    2568 generic.go:358] "Generic (PLEG): container finished" podID="355533f6-8ffa-4295-b679-074b729f11ba" containerID="09bb161e8966c6f31bbe83610b0ffc54bbcfdd9df9b868d270f8b8d1d8b8b6c4" exitCode=7
May 28 20:15:03.173121 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:03.173074    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" event={"ID":"355533f6-8ffa-4295-b679-074b729f11ba","Type":"ContainerDied","Data":"09bb161e8966c6f31bbe83610b0ffc54bbcfdd9df9b868d270f8b8d1d8b8b6c4"}
May 28 20:15:03.173121 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:03.173114    2568 scope.go:117] "RemoveContainer" containerID="8ab2d1d5671168845b425d1e892024f7e0f8cd2b0c0d8b7e03896481670d00ed"
May 28 20:15:03.173674 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:03.173351    2568 scope.go:117] "RemoveContainer" containerID="09bb161e8966c6f31bbe83610b0ffc54bbcfdd9df9b868d270f8b8d1d8b8b6c4"
May 28 20:15:03.173674 ip-10-0-135-191 kubenswrapper[2568]: E0528 20:15:03.173646    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29666655-zdchs_opendatahub(355533f6-8ffa-4295-b679-074b729f11ba)\"" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" podUID="355533f6-8ffa-4295-b679-074b729f11ba"
May 28 20:15:04.178872 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:04.178842    2568 scope.go:117] "RemoveContainer" containerID="09bb161e8966c6f31bbe83610b0ffc54bbcfdd9df9b868d270f8b8d1d8b8b6c4"
May 28 20:15:04.179286 ip-10-0-135-191 kubenswrapper[2568]: E0528 20:15:04.179050    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29666655-zdchs_opendatahub(355533f6-8ffa-4295-b679-074b729f11ba)\"" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" podUID="355533f6-8ffa-4295-b679-074b729f11ba"
May 28 20:15:17.695659 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:17.695621    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-tfpxs_655333f5-f77e-4b0d-b0a1-2381f51373c8/manager/0.log"
May 28 20:15:17.900580 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:17.900551    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29666655-zdchs_355533f6-8ffa-4295-b679-074b729f11ba/cleanup/1.log"
May 28 20:15:18.118978 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:18.118941    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-rjql7_87f1d890-260d-41b4-8453-2874a5269507/manager/2.log"
May 28 20:15:18.222160 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:18.222131    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69568b6749-c285w_752b8b27-f963-45a7-b7fd-d677aba0da51/manager/0.log"
May 28 20:15:18.564110 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:18.564073    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-fldds_b41c230d-5c52-4ecd-a0e2-a9487b7dd896/postgres/0.log"
May 28 20:15:19.330628 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:19.330590    2568 scope.go:117] "RemoveContainer" containerID="09bb161e8966c6f31bbe83610b0ffc54bbcfdd9df9b868d270f8b8d1d8b8b6c4"
May 28 20:15:19.866410 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:19.866322    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-kcmfw_0f65f48f-e9b2-4495-bcf4-efe039dc953a/manager/0.log"
May 28 20:15:20.234015 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:20.233931    2568 generic.go:358] "Generic (PLEG): container finished" podID="355533f6-8ffa-4295-b679-074b729f11ba" containerID="f967a493ed09c36af7b97bd01c366c6e53ad6e00c7df94f40f0154980b33f481" exitCode=7
May 28 20:15:20.234015 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:20.234001    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" event={"ID":"355533f6-8ffa-4295-b679-074b729f11ba","Type":"ContainerDied","Data":"f967a493ed09c36af7b97bd01c366c6e53ad6e00c7df94f40f0154980b33f481"}
May 28 20:15:20.234210 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:20.234049    2568 scope.go:117] "RemoveContainer" containerID="09bb161e8966c6f31bbe83610b0ffc54bbcfdd9df9b868d270f8b8d1d8b8b6c4"
May 28 20:15:20.234419 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:20.234394    2568 scope.go:117] "RemoveContainer" containerID="f967a493ed09c36af7b97bd01c366c6e53ad6e00c7df94f40f0154980b33f481"
May 28 20:15:20.234694 ip-10-0-135-191 kubenswrapper[2568]: E0528 20:15:20.234676    2568 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29666655-zdchs_opendatahub(355533f6-8ffa-4295-b679-074b729f11ba)\"" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" podUID="355533f6-8ffa-4295-b679-074b729f11ba"
May 28 20:15:20.958931 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:20.958901    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-2fgcr_30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89/discovery/0.log"
May 28 20:15:21.163248 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.163216    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58474d499-pvr2w_1b40d81b-7ffb-4649-9271-eb495e680d77/kube-auth-proxy/0.log"
May 28 20:15:21.265794 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.265708    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666655-zdchs"]
May 28 20:15:21.407619 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.407595    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:21.492075 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.492046    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c59b4dd8-8lv8f_e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54/router/0.log"
May 28 20:15:21.558863 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.558828    2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qr526\" (UniqueName: \"kubernetes.io/projected/355533f6-8ffa-4295-b679-074b729f11ba-kube-api-access-qr526\") pod \"355533f6-8ffa-4295-b679-074b729f11ba\" (UID: \"355533f6-8ffa-4295-b679-074b729f11ba\") "
May 28 20:15:21.561082 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.561055    2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/355533f6-8ffa-4295-b679-074b729f11ba-kube-api-access-qr526" (OuterVolumeSpecName: "kube-api-access-qr526") pod "355533f6-8ffa-4295-b679-074b729f11ba" (UID: "355533f6-8ffa-4295-b679-074b729f11ba"). InnerVolumeSpecName "kube-api-access-qr526". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 20:15:21.660031 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.659999    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr526\" (UniqueName: \"kubernetes.io/projected/355533f6-8ffa-4295-b679-074b729f11ba-kube-api-access-qr526\") on node \"ip-10-0-135-191.ec2.internal\" DevicePath \"\""
May 28 20:15:21.819852 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.819767    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9_dd87511a-849c-4ab3-8fff-ade9399f0e15/storage-initializer/0.log"
May 28 20:15:21.826014 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:21.825980    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-pt5g9_dd87511a-849c-4ab3-8fff-ade9399f0e15/main/0.log"
May 28 20:15:22.243955 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.243857    2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs"
May 28 20:15:22.243955 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.243878    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29666655-zdchs" event={"ID":"355533f6-8ffa-4295-b679-074b729f11ba","Type":"ContainerDied","Data":"8375630c5198eaef89d7cd550256582e3c4e7ff3242103bbc4802f6c5027e2ff"}
May 28 20:15:22.243955 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.243932    2568 scope.go:117] "RemoveContainer" containerID="f967a493ed09c36af7b97bd01c366c6e53ad6e00c7df94f40f0154980b33f481"
May 28 20:15:22.265119 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.265086    2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666655-zdchs"]
May 28 20:15:22.273675 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.273647    2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-29666655-zdchs"]
May 28 20:15:22.273813 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.273661    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl_f812fb9c-5011-4ca0-9b29-cef2e22672e6/storage-initializer/0.log"
May 28 20:15:22.280340 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.280321    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-xqjpl_f812fb9c-5011-4ca0-9b29-cef2e22672e6/main/0.log"
May 28 20:15:22.392208 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.392177    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-jnbct_6c0699f6-268c-4e22-8a6a-9ef6372bc287/storage-initializer/0.log"
May 28 20:15:22.398904 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:22.398882    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-jnbct_6c0699f6-268c-4e22-8a6a-9ef6372bc287/main/0.log"
May 28 20:15:23.338577 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:23.338545    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="355533f6-8ffa-4295-b679-074b729f11ba" path="/var/lib/kubelet/pods/355533f6-8ffa-4295-b679-074b729f11ba/volumes"
May 28 20:15:29.765492 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:29.765441    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-px2n9_11c4e6c3-c807-4c58-bf0a-551c264cb561/global-pull-secret-syncer/0.log"
May 28 20:15:29.847371 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:29.847334    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c5fpz_097c5ace-0c7c-495a-8e67-c9222b086068/konnectivity-agent/0.log"
May 28 20:15:29.925241 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:29.925208    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-191.ec2.internal_f0dac381d94558e70808c6353a7e9fc8/haproxy/0.log"
May 28 20:15:33.553380 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:33.553349    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-kcmfw_0f65f48f-e9b2-4495-bcf4-efe039dc953a/manager/0.log"
May 28 20:15:35.070140 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:35.070108    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-mq7cf_561fea3e-dfb5-42b2-b1d1-0581fec1d6eb/cluster-monitoring-operator/0.log"
May 28 20:15:35.390837 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:35.390768    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mqjcz_d9c58404-d17e-42d4-9d1e-982278a6606a/node-exporter/0.log"
May 28 20:15:35.413005 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:35.412977    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mqjcz_d9c58404-d17e-42d4-9d1e-982278a6606a/kube-rbac-proxy/0.log"
May 28 20:15:35.435462 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:35.435437    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mqjcz_d9c58404-d17e-42d4-9d1e-982278a6606a/init-textfile/0.log"
May 28 20:15:37.757095 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:37.757065    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/1.log"
May 28 20:15:37.761161 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:37.761135    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-9vlfb_5dc02322-296b-443f-bb2b-900dbdd4a4c8/console-operator/2.log"
May 28 20:15:38.682931 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.682894    2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"]
May 28 20:15:38.683237 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683224    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.683280 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683240    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.683280 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683260    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.683280 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683265    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.683388 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683371    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.683388 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683381    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.683388 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.683387    2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="355533f6-8ffa-4295-b679-074b729f11ba" containerName="cleanup"
May 28 20:15:38.686487 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.686441    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.688990 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.688958    2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-74sxj\"/\"default-dockercfg-tqdrh\""
May 28 20:15:38.690045 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.690026    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-74sxj\"/\"openshift-service-ca.crt\""
May 28 20:15:38.690175 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.690030    2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-74sxj\"/\"kube-root-ca.crt\""
May 28 20:15:38.696414 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.696384    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"]
May 28 20:15:38.808058 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.808028    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-mgxxg_47aee4c8-e1ee-4f87-9e23-63f3146f9608/volume-data-source-validator/0.log"
May 28 20:15:38.818939 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.818914    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-proc\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.819068 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.818951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-lib-modules\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.819068 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.819031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-podres\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.819136 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.819092    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-sys\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.819136 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.819129    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf854\" (UniqueName: \"kubernetes.io/projected/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-kube-api-access-gf854\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920503 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920432    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-proc\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920517    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-lib-modules\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920568    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-proc\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920574    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-podres\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920633    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-sys\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920673    2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gf854\" (UniqueName: \"kubernetes.io/projected/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-kube-api-access-gf854\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920675    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-podres\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.920712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920677    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-lib-modules\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.921034 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.920747    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-sys\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.928456 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.928430    2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf854\" (UniqueName: \"kubernetes.io/projected/a4362b1e-d4b3-4b2d-899e-f4c45575d4b0-kube-api-access-gf854\") pod \"perf-node-gather-daemonset-nspcs\" (UID: \"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0\") " pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:38.998333 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:38.998220    2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:39.134518 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.134484    2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"]
May 28 20:15:39.137371 ip-10-0-135-191 kubenswrapper[2568]: W0528 20:15:39.137338    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda4362b1e_d4b3_4b2d_899e_f4c45575d4b0.slice/crio-f41cea90b8cc1f802a68b3534cfc749260b3f3a4e8ea17cc790dcc983d75bf66 WatchSource:0}: Error finding container f41cea90b8cc1f802a68b3534cfc749260b3f3a4e8ea17cc790dcc983d75bf66: Status 404 returned error can't find the container with id f41cea90b8cc1f802a68b3534cfc749260b3f3a4e8ea17cc790dcc983d75bf66
May 28 20:15:39.307947 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.307903    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs" event={"ID":"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0","Type":"ContainerStarted","Data":"beaee0df3a3e403317a93021fde6afdb42f8ff25ae6debb43fe5e9bc04d51a15"}
May 28 20:15:39.307947 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.307942    2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs" event={"ID":"a4362b1e-d4b3-4b2d-899e-f4c45575d4b0","Type":"ContainerStarted","Data":"f41cea90b8cc1f802a68b3534cfc749260b3f3a4e8ea17cc790dcc983d75bf66"}
May 28 20:15:39.308200 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.308005    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:39.323072 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.323017    2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs" podStartSLOduration=1.322999395 podStartE2EDuration="1.322999395s" podCreationTimestamp="2026-05-28 20:15:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:15:39.321855635 +0000 UTC m=+1952.609196551" watchObservedRunningTime="2026-05-28 20:15:39.322999395 +0000 UTC m=+1952.610340298"
May 28 20:15:39.678922 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.678837    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-489wq_d9ecf972-8310-46b3-93b8-37284fcd1e04/dns/0.log"
May 28 20:15:39.699673 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.699646    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-489wq_d9ecf972-8310-46b3-93b8-37284fcd1e04/kube-rbac-proxy/0.log"
May 28 20:15:39.805712 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:39.805678    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rflwq_054a4749-b39d-4f52-b4cc-cc15a9a93bf8/dns-node-resolver/0.log"
May 28 20:15:40.398118 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:40.398092    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hl88v_bd7826c6-6b85-45d0-9a3a-c1b4fca63208/node-ca/0.log"
May 28 20:15:41.268366 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:41.268330    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-2fgcr_30e3f39e-fbb7-4fdf-8cc3-e1800b32ff89/discovery/0.log"
May 28 20:15:41.308315 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:41.308290    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58474d499-pvr2w_1b40d81b-7ffb-4649-9271-eb495e680d77/kube-auth-proxy/0.log"
May 28 20:15:41.423263 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:41.423227    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58c59b4dd8-8lv8f_e2acbbe9-dff4-4b15-be5c-7eff8e9fdd54/router/0.log"
May 28 20:15:41.954095 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:41.954066    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lvcmf_0c57594e-7784-4885-ab2f-577790157a3f/serve-healthcheck-canary/0.log"
May 28 20:15:42.449106 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:42.449067    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-2t85s_63e20a96-5b18-4200-99ee-4d2d35606b95/insights-operator/0.log"
May 28 20:15:42.449525 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:42.449223    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-2t85s_63e20a96-5b18-4200-99ee-4d2d35606b95/insights-operator/1.log"
May 28 20:15:42.469844 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:42.469816    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lh4zd_9336ef2c-04f1-4358-965f-12e45660a144/kube-rbac-proxy/0.log"
May 28 20:15:42.488738 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:42.488714    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lh4zd_9336ef2c-04f1-4358-965f-12e45660a144/exporter/0.log"
May 28 20:15:42.508139 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:42.508110    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lh4zd_9336ef2c-04f1-4358-965f-12e45660a144/extractor/0.log"
May 28 20:15:44.437855 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:44.437825    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-tfpxs_655333f5-f77e-4b0d-b0a1-2381f51373c8/manager/0.log"
May 28 20:15:44.501345 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:44.501319    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-rjql7_87f1d890-260d-41b4-8453-2874a5269507/manager/1.log"
May 28 20:15:44.511536 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:44.511510    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-rjql7_87f1d890-260d-41b4-8453-2874a5269507/manager/2.log"
May 28 20:15:44.532077 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:44.532050    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69568b6749-c285w_752b8b27-f963-45a7-b7fd-d677aba0da51/manager/0.log"
May 28 20:15:44.618945 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:44.618918    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-fldds_b41c230d-5c52-4ecd-a0e2-a9487b7dd896/postgres/0.log"
May 28 20:15:45.322562 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:45.322533    2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-74sxj/perf-node-gather-daemonset-nspcs"
May 28 20:15:50.424068 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:50.424038    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-r5f46_bda09b67-e01f-4af9-91a4-6aa913ab86d8/kube-storage-version-migrator-operator/1.log"
May 28 20:15:50.424567 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:50.424551    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-r5f46_bda09b67-e01f-4af9-91a4-6aa913ab86d8/kube-storage-version-migrator-operator/0.log"
May 28 20:15:51.343440 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.343412    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/kube-multus-additional-cni-plugins/0.log"
May 28 20:15:51.364247 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.364213    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/egress-router-binary-copy/0.log"
May 28 20:15:51.389298 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.389271    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/cni-plugins/0.log"
May 28 20:15:51.409830 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.409802    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/bond-cni-plugin/0.log"
May 28 20:15:51.430845 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.430813    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/routeoverride-cni/0.log"
May 28 20:15:51.452899 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.452870    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/whereabouts-cni-bincopy/0.log"
May 28 20:15:51.473574 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.473538    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j7t6m_615454c2-0260-4ded-a724-210769c2ebfc/whereabouts-cni/0.log"
May 28 20:15:51.813578 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.813544    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ngpzm_b81ecf59-2f24-43f5-a413-87f270208d8d/kube-multus/0.log"
May 28 20:15:51.939772 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.939739    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wxzjd_0f40971c-dd1a-4403-a51b-bdd13d1fb3cc/network-metrics-daemon/0.log"
May 28 20:15:51.957972 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:51.957945    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-wxzjd_0f40971c-dd1a-4403-a51b-bdd13d1fb3cc/kube-rbac-proxy/0.log"
May 28 20:15:53.009997 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.009961    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-controller/0.log"
May 28 20:15:53.030893 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.030855    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/0.log"
May 28 20:15:53.040489 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.040433    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovn-acl-logging/1.log"
May 28 20:15:53.059804 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.059772    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/kube-rbac-proxy-node/0.log"
May 28 20:15:53.081602 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.081569    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/kube-rbac-proxy-ovn-metrics/0.log"
May 28 20:15:53.098369 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.098340    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/northd/0.log"
May 28 20:15:53.116715 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.116687    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/nbdb/0.log"
May 28 20:15:53.135570 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.135546    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/sbdb/0.log"
May 28 20:15:53.233426 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:53.233393    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rxvsp_080ace10-d27e-47de-a72c-67815583d217/ovnkube-controller/0.log"
May 28 20:15:54.614790 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:54.614764    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7f44b6dd87-7fmqd_a09f5486-a17a-4554-a6d6-af3c724b747e/check-endpoints/0.log"
May 28 20:15:54.677657 ip-10-0-135-191 kubenswrapper[2568]: I0528 20:15:54.677625    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-r9l4n_e9e44915-c783-4fb0-84e0-1b0c0d67dfd2/network-check-target-container/0.log"