Apr 20 21:44:36.367015 ip-10-0-136-102 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Apr 20 21:44:36.367031 ip-10-0-136-102 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Apr 20 21:44:36.367041 ip-10-0-136-102 systemd[1]: kubelet.service: Failed with result 'resources'. Apr 20 21:44:36.367428 ip-10-0-136-102 systemd[1]: Failed to start Kubernetes Kubelet. Apr 20 21:44:47.865465 ip-10-0-136-102 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Apr 20 21:44:47.865486 ip-10-0-136-102 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cd4fb82498f4487db5d654b89cce9d77 -- Apr 20 21:47:12.704947 ip-10-0-136-102 systemd[1]: Starting Kubernetes Kubelet... Apr 20 21:47:13.197407 ip-10-0-136-102 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 20 21:47:13.197407 ip-10-0-136-102 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 20 21:47:13.197407 ip-10-0-136-102 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 20 21:47:13.197407 ip-10-0-136-102 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 20 21:47:13.197407 ip-10-0-136-102 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 20 21:47:13.199481 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.199376 2576 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204014 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204040 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204046 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204050 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204055 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204059 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204063 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 20 21:47:13.204056 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204068 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204073 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204077 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204081 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204084 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204088 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204092 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204096 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204100 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204104 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204107 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204111 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204114 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204118 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204124 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204130 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204134 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204139 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204143 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204147 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 20 21:47:13.204505 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204151 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204155 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204161 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204164 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204168 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204172 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204175 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204179 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204183 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204187 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204191 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204194 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204198 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204202 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204207 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204221 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204226 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204230 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204234 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204238 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 20 21:47:13.205334 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204243 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204248 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204252 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204259 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204266 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204270 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204274 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204279 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204283 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204287 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204291 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204296 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204300 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204304 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204308 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204312 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204316 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204322 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204326 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 20 21:47:13.205979 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204331 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204335 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204339 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204343 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204348 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204352 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204356 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204361 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204365 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204370 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204374 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204378 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204385 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204389 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204394 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204398 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204402 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204406 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204410 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.204415 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 20 21:47:13.206460 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205115 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205124 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205130 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205134 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205139 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205143 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205149 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205153 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205158 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205162 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205167 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205171 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205176 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205180 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205184 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205189 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205193 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205197 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205201 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205205 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 20 21:47:13.206987 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205209 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205213 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205218 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205222 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205227 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205231 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205236 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205240 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205243 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205247 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205252 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205256 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205261 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205265 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205269 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205273 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205278 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205282 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205286 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205290 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 20 21:47:13.207947 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205302 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205307 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205311 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205315 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205319 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205323 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205327 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205332 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205336 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205339 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205344 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205348 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205352 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205356 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205360 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205364 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205369 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205374 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205379 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205383 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 20 21:47:13.208692 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205387 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205391 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205395 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205400 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205404 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205408 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205412 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205417 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205424 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205430 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205436 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205441 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205452 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205457 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205461 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205466 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205471 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205475 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205479 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 20 21:47:13.209191 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205483 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205487 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205493 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205498 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205502 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205506 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.205511 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.206942 2576 flags.go:64] FLAG: --address="0.0.0.0" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.206961 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.206974 2576 flags.go:64] FLAG: --anonymous-auth="true" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.206981 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.206991 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.206996 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207002 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207011 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207017 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207022 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207028 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207034 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207039 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207045 2576 flags.go:64] FLAG: --cgroup-root="" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207049 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207054 2576 flags.go:64] FLAG: --client-ca-file="" Apr 20 21:47:13.209785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207059 2576 flags.go:64] FLAG: --cloud-config="" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207064 2576 flags.go:64] FLAG: --cloud-provider="external" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207069 2576 flags.go:64] FLAG: --cluster-dns="[]" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207076 2576 flags.go:64] FLAG: --cluster-domain="" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207080 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207085 2576 flags.go:64] FLAG: --config-dir="" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207090 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207095 2576 flags.go:64] FLAG: --container-log-max-files="5" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207110 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207115 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207120 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207125 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207130 2576 flags.go:64] FLAG: --contention-profiling="false" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207134 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207139 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207145 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207150 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207157 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207162 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207167 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207172 2576 flags.go:64] FLAG: --enable-load-reader="false" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207178 2576 flags.go:64] FLAG: --enable-server="true" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207183 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207191 2576 flags.go:64] FLAG: --event-burst="100" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207196 2576 flags.go:64] FLAG: --event-qps="50" Apr 20 21:47:13.210577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207200 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207204 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207209 2576 flags.go:64] FLAG: --eviction-hard="" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207215 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207219 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207224 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207229 2576 flags.go:64] FLAG: --eviction-soft="" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207233 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207238 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207243 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207248 2576 flags.go:64] FLAG: --experimental-mounter-path="" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207253 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207258 2576 flags.go:64] FLAG: --fail-swap-on="true" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207263 2576 flags.go:64] FLAG: --feature-gates="" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207269 2576 flags.go:64] FLAG: --file-check-frequency="20s" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207274 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207280 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207285 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207290 2576 flags.go:64] FLAG: --healthz-port="10248" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207295 2576 flags.go:64] FLAG: --help="false" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207300 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207305 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207310 2576 flags.go:64] FLAG: --http-check-frequency="20s" Apr 20 21:47:13.211230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207315 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207321 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207327 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207332 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207337 2576 flags.go:64] FLAG: --image-service-endpoint="" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207342 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207348 2576 flags.go:64] FLAG: --kube-api-burst="100" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207353 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207358 2576 flags.go:64] FLAG: --kube-api-qps="50" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207363 2576 flags.go:64] FLAG: --kube-reserved="" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207368 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207373 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207378 2576 flags.go:64] FLAG: --kubelet-cgroups="" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207383 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207387 2576 flags.go:64] FLAG: --lock-file="" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207392 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207397 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207403 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207412 2576 flags.go:64] FLAG: --log-json-split-stream="false" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207417 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207422 2576 flags.go:64] FLAG: --log-text-split-stream="false" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207427 2576 flags.go:64] FLAG: --logging-format="text" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207431 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207437 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 20 21:47:13.211916 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207442 2576 flags.go:64] FLAG: --manifest-url="" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207447 2576 flags.go:64] FLAG: --manifest-url-header="" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207454 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207459 2576 flags.go:64] FLAG: --max-open-files="1000000" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207466 2576 flags.go:64] FLAG: --max-pods="110" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207471 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207475 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207480 2576 flags.go:64] FLAG: --memory-manager-policy="None" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207485 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207490 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207495 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207500 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207513 2576 flags.go:64] FLAG: --node-status-max-images="50" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207518 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207523 2576 flags.go:64] FLAG: --oom-score-adj="-999" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207529 2576 flags.go:64] FLAG: --pod-cidr="" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207533 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c8cfe89231412ff3ee8cb6207fa0be33cad0f08e88c9c0f1e9f7e8c6f14d6715" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207542 2576 flags.go:64] FLAG: --pod-manifest-path="" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207546 2576 flags.go:64] FLAG: --pod-max-pids="-1" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207551 2576 flags.go:64] FLAG: --pods-per-core="0" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207556 2576 flags.go:64] FLAG: --port="10250" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207561 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207565 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0140fe46879603b00" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207570 2576 flags.go:64] FLAG: --qos-reserved="" Apr 20 21:47:13.212507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207576 2576 flags.go:64] FLAG: --read-only-port="10255" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207581 2576 flags.go:64] FLAG: --register-node="true" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207585 2576 flags.go:64] FLAG: --register-schedulable="true" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207615 2576 flags.go:64] FLAG: --register-with-taints="" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207652 2576 flags.go:64] FLAG: --registry-burst="10" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207659 2576 flags.go:64] FLAG: --registry-qps="5" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207664 2576 flags.go:64] FLAG: --reserved-cpus="" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207669 2576 flags.go:64] FLAG: --reserved-memory="" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207679 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207684 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207689 2576 flags.go:64] FLAG: --rotate-certificates="false" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207694 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207699 2576 flags.go:64] FLAG: --runonce="false" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207704 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207709 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207714 2576 flags.go:64] FLAG: --seccomp-default="false" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207719 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207723 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207729 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207734 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207739 2576 flags.go:64] FLAG: --storage-driver-password="root" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207744 2576 flags.go:64] FLAG: --storage-driver-secure="false" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207748 2576 flags.go:64] FLAG: --storage-driver-table="stats" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207753 2576 flags.go:64] FLAG: --storage-driver-user="root" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207760 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207766 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 20 21:47:13.213113 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207771 2576 flags.go:64] FLAG: --system-cgroups="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207775 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207785 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207789 2576 flags.go:64] FLAG: --tls-cert-file="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207793 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207801 2576 flags.go:64] FLAG: --tls-min-version="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207806 2576 flags.go:64] FLAG: --tls-private-key-file="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207811 2576 flags.go:64] FLAG: --topology-manager-policy="none" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207816 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207823 2576 flags.go:64] FLAG: --topology-manager-scope="container" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207834 2576 flags.go:64] FLAG: --v="2" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207841 2576 flags.go:64] FLAG: --version="false" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207848 2576 flags.go:64] FLAG: --vmodule="" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207856 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.207861 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208021 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208030 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208035 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208039 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208044 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208048 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208052 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208057 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 20 21:47:13.213796 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208061 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208066 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208070 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208074 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208078 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208083 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208087 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208091 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208097 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208102 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208106 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208110 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208114 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208119 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208123 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208127 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208131 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208135 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208141 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208147 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 20 21:47:13.214350 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208152 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208156 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208161 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208165 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208169 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208173 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208177 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208182 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208188 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208193 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208197 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208202 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208206 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208210 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208214 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208218 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208223 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208227 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208231 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 20 21:47:13.214950 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208235 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208239 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208244 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208248 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208253 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208257 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208261 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208265 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208269 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208273 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208279 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208286 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208293 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208297 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208302 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208306 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208311 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208316 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208320 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208325 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 20 21:47:13.215431 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208329 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208334 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208338 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208343 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208347 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208351 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208355 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208360 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208364 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208368 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208372 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208376 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208380 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208384 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208389 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208394 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208398 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208402 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 20 21:47:13.215949 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.208406 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.209257 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.216166 2576 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.216298 2576 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216352 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216357 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216361 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216364 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216367 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216371 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216374 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216377 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216380 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216383 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216386 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 20 21:47:13.216410 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216389 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216393 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216398 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216401 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216404 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216406 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216409 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216413 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216417 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216421 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216424 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216426 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216429 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216432 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216437 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216440 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216443 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216446 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 20 21:47:13.216815 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216449 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216452 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216454 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216457 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216460 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216462 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216465 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216468 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216471 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216473 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216476 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216478 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216481 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216483 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216486 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216489 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216492 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216495 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216498 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216500 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 20 21:47:13.217284 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216503 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216505 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216508 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216511 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216513 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216516 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216518 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216521 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216523 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216526 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216529 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216531 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216534 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216537 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216539 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216542 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216544 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216547 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216550 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216552 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 20 21:47:13.217826 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216555 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216557 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216560 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216562 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216565 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216567 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216570 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216573 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216577 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216580 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216583 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216585 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216588 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216590 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216593 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216595 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 20 21:47:13.218320 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216598 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.216603 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216748 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216753 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216756 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216759 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216767 2576 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216770 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216773 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216776 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216778 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216781 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216783 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216786 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216789 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 20 21:47:13.218740 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216792 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216796 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216799 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216802 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216804 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216807 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216809 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216812 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216815 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216818 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216821 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216826 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216830 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216833 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216836 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216839 2576 feature_gate.go:328] unrecognized feature gate: Example2 Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216842 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216845 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216848 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216851 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 20 21:47:13.219119 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216853 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216856 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216859 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216861 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216865 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216867 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216870 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216873 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216875 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216878 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216880 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216883 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216885 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216888 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216891 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216893 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216896 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216899 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216901 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216904 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 20 21:47:13.219610 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216906 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216909 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216912 2576 feature_gate.go:328] unrecognized feature gate: Example Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216914 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216917 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216920 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216923 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216925 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216928 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216930 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216933 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216935 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216938 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216940 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216943 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216945 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216948 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216951 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216953 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 20 21:47:13.220116 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216956 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216958 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216961 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216963 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216966 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216969 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216971 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216974 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216977 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216979 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216982 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216984 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216987 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:13.216989 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.216994 2576 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 20 21:47:13.220588 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.217981 2576 server.go:962] "Client rotation is on, will bootstrap in background" Apr 20 21:47:13.222331 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.222314 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 20 21:47:13.223518 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.223506 2576 server.go:1019] "Starting client certificate rotation" Apr 20 21:47:13.223639 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.223609 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 20 21:47:13.223676 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.223666 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 20 21:47:13.253381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.253354 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 20 21:47:13.256579 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.256551 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 20 21:47:13.277263 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.277232 2576 log.go:25] "Validated CRI v1 runtime API" Apr 20 21:47:13.282287 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.282253 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 20 21:47:13.283459 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.283443 2576 log.go:25] "Validated CRI v1 image API" Apr 20 21:47:13.285328 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.285305 2576 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 20 21:47:13.289466 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.289441 2576 fs.go:135] Filesystem UUIDs: map[320a5c60-ace9-47ab-9214-5433e8630e90:/dev/nvme0n1p4 5a5c15bd-20d5-47f0-892a-e82abf2828e8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Apr 20 21:47:13.289552 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.289464 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 20 21:47:13.295284 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.295157 2576 manager.go:217] Machine: {Timestamp:2026-04-20 21:47:13.293958479 +0000 UTC m=+0.461279217 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100313 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24b3794dd874129e1c758e58777398 SystemUUID:ec24b379-4dd8-7412-9e1c-758e58777398 BootID:cd4fb824-98f4-487d-b5d6-54b89cce9d77 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dd:f9:d2:fd:5d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dd:f9:d2:fd:5d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:e4:ea:55:3e:ec Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 20 21:47:13.295284 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.295277 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 20 21:47:13.295431 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.295419 2576 manager.go:233] Version: {KernelVersion:5.14.0-570.107.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260414-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 20 21:47:13.296669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.296641 2576 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 20 21:47:13.296838 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.296671 2576 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 20 21:47:13.296886 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.296850 2576 topology_manager.go:138] "Creating topology manager with none policy" Apr 20 21:47:13.296886 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.296859 2576 container_manager_linux.go:306] "Creating device plugin manager" Apr 20 21:47:13.296886 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.296873 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 20 21:47:13.296886 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.296885 2576 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 20 21:47:13.297186 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.297170 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p5nwk" Apr 20 21:47:13.298322 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.298310 2576 state_mem.go:36] "Initialized new in-memory state store" Apr 20 21:47:13.298466 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.298456 2576 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 20 21:47:13.300941 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.300928 2576 kubelet.go:491] "Attempting to sync node with API server" Apr 20 21:47:13.300983 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.300948 2576 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 20 21:47:13.300983 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.300962 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 20 21:47:13.300983 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.300971 2576 kubelet.go:397] "Adding apiserver pod source" Apr 20 21:47:13.300983 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.300981 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 20 21:47:13.302221 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.302206 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 20 21:47:13.302272 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.302233 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 20 21:47:13.304307 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.304289 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p5nwk" Apr 20 21:47:13.305951 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.305851 2576 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 20 21:47:13.308227 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.308195 2576 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 20 21:47:13.310506 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310485 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 20 21:47:13.310506 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310508 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310515 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310521 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310527 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310533 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310539 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310545 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310552 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310559 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310567 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 20 21:47:13.310620 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.310577 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 20 21:47:13.311553 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.311539 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 20 21:47:13.311587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.311554 2576 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 20 21:47:13.313789 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.313771 2576 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:13.315239 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.315224 2576 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 20 21:47:13.315331 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.315264 2576 server.go:1295] "Started kubelet" Apr 20 21:47:13.315397 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.315357 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 20 21:47:13.315442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.315367 2576 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 20 21:47:13.315442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.315437 2576 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 20 21:47:13.316215 ip-10-0-136-102 systemd[1]: Started Kubernetes Kubelet. Apr 20 21:47:13.317458 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.317443 2576 server.go:317] "Adding debug handlers to kubelet server" Apr 20 21:47:13.317573 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.317445 2576 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:13.318405 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.318392 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-102.ec2.internal" not found Apr 20 21:47:13.319108 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.319082 2576 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 20 21:47:13.324340 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.324318 2576 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 20 21:47:13.324458 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.324375 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 20 21:47:13.324918 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.324886 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 20 21:47:13.325682 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325661 2576 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325686 2576 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.325692 2576 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-102.ec2.internal\" not found" Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325661 2576 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325727 2576 factory.go:153] Registering CRI-O factory Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325776 2576 reconstruct.go:97] "Volume reconstruction finished" Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325783 2576 reconciler.go:26] "Reconciler: start to sync state" Apr 20 21:47:13.325797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325776 2576 factory.go:223] Registration of the crio container factory successfully Apr 20 21:47:13.326116 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325847 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 20 21:47:13.326116 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325855 2576 factory.go:55] Registering systemd factory Apr 20 21:47:13.326116 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325861 2576 factory.go:223] Registration of the systemd container factory successfully Apr 20 21:47:13.326116 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325879 2576 factory.go:103] Registering Raw factory Apr 20 21:47:13.326116 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.325890 2576 manager.go:1196] Started watching for new ooms in manager Apr 20 21:47:13.326904 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.326890 2576 manager.go:319] Starting recovery of all containers Apr 20 21:47:13.327228 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.327210 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:13.333664 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.333574 2576 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-102.ec2.internal\" not found" node="ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.335897 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.335873 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-102.ec2.internal" not found Apr 20 21:47:13.337222 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.337205 2576 manager.go:324] Recovery completed Apr 20 21:47:13.341584 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.341506 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 20 21:47:13.343399 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.343383 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeHasSufficientMemory" Apr 20 21:47:13.343487 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.343412 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeHasNoDiskPressure" Apr 20 21:47:13.343487 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.343423 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeHasSufficientPID" Apr 20 21:47:13.343940 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.343922 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 20 21:47:13.343940 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.343940 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 20 21:47:13.344066 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.343957 2576 state_mem.go:36] "Initialized new in-memory state store" Apr 20 21:47:13.346454 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.346440 2576 policy_none.go:49] "None policy: Start" Apr 20 21:47:13.346511 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.346460 2576 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 20 21:47:13.346511 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.346470 2576 state_mem.go:35] "Initializing new in-memory state store" Apr 20 21:47:13.386415 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386395 2576 manager.go:341] "Starting Device Plugin manager" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.386450 2576 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386464 2576 server.go:85] "Starting device plugin registration server" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386767 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386778 2576 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386877 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386961 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.386968 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.387992 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.388033 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-102.ec2.internal\" not found" Apr 20 21:47:13.408029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.390815 2576 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-102.ec2.internal" not found Apr 20 21:47:13.463917 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.463823 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 20 21:47:13.465160 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.465140 2576 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 20 21:47:13.465239 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.465177 2576 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 20 21:47:13.465239 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.465201 2576 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 20 21:47:13.465239 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.465210 2576 kubelet.go:2451] "Starting kubelet main sync loop" Apr 20 21:47:13.465394 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:13.465318 2576 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 20 21:47:13.469800 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.469778 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:13.487715 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.487689 2576 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 20 21:47:13.488731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.488709 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeHasSufficientMemory" Apr 20 21:47:13.488847 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.488747 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeHasNoDiskPressure" Apr 20 21:47:13.488847 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.488763 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeHasSufficientPID" Apr 20 21:47:13.488847 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.488795 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.497865 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.497840 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.566122 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.566078 2576 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal"] Apr 20 21:47:13.570556 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.570537 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.570654 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.570549 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.598509 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.598480 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.601954 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.601936 2576 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.608821 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.608803 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 20 21:47:13.615117 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.615099 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 20 21:47:13.627406 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.627383 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3588c42b05d40fa47c543e3dbe98316-config\") pod \"kube-apiserver-proxy-ip-10-0-136-102.ec2.internal\" (UID: \"f3588c42b05d40fa47c543e3dbe98316\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.627485 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.627414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eba8a1e795b921c317949461306d2685-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal\" (UID: \"eba8a1e795b921c317949461306d2685\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.627485 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.627434 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eba8a1e795b921c317949461306d2685-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal\" (UID: \"eba8a1e795b921c317949461306d2685\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.727956 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.727875 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eba8a1e795b921c317949461306d2685-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal\" (UID: \"eba8a1e795b921c317949461306d2685\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.727956 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.727912 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eba8a1e795b921c317949461306d2685-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal\" (UID: \"eba8a1e795b921c317949461306d2685\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.727956 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.727933 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3588c42b05d40fa47c543e3dbe98316-config\") pod \"kube-apiserver-proxy-ip-10-0-136-102.ec2.internal\" (UID: \"f3588c42b05d40fa47c543e3dbe98316\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.728132 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.727962 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eba8a1e795b921c317949461306d2685-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal\" (UID: \"eba8a1e795b921c317949461306d2685\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.728132 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.727995 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eba8a1e795b921c317949461306d2685-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal\" (UID: \"eba8a1e795b921c317949461306d2685\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.728132 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.728013 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f3588c42b05d40fa47c543e3dbe98316-config\") pod \"kube-apiserver-proxy-ip-10-0-136-102.ec2.internal\" (UID: \"f3588c42b05d40fa47c543e3dbe98316\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.911951 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.911915 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" Apr 20 21:47:13.917491 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:13.917463 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" Apr 20 21:47:14.224579 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.224549 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 20 21:47:14.225358 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.224753 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 20 21:47:14.225358 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.224767 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 20 21:47:14.225358 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.224756 2576 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 20 21:47:14.301442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.301411 2576 apiserver.go:52] "Watching apiserver" Apr 20 21:47:14.307766 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.307732 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-19 21:42:13 +0000 UTC" deadline="2028-01-25 15:18:41.513889816 +0000 UTC" Apr 20 21:47:14.307766 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.307761 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15473h31m27.206132158s" Apr 20 21:47:14.309084 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.309064 2576 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 20 21:47:14.313272 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.312990 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-7b48f","openshift-network-operator/iptables-alerter-j45q2","openshift-ovn-kubernetes/ovnkube-node-vf7zb","kube-system/konnectivity-agent-thbph","kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal","openshift-multus/multus-4c5hz","openshift-network-diagnostics/network-check-target-wnz85","openshift-cluster-node-tuning-operator/tuned-dclhd","openshift-dns/node-resolver-7t457","openshift-image-registry/node-ca-fhmlz","openshift-multus/multus-additional-cni-plugins-cm52p"] Apr 20 21:47:14.316792 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.316768 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.316913 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.316852 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:14.319167 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.319145 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.321934 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.321912 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.322030 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.322002 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 20 21:47:14.322077 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.322022 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.322077 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.322052 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.322163 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.322058 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-22dw7\"" Apr 20 21:47:14.324533 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.324480 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 20 21:47:14.324533 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.324512 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 20 21:47:14.324733 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.324588 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.326041 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326007 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 20 21:47:14.326041 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326019 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 20 21:47:14.326263 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326046 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wj45z\"" Apr 20 21:47:14.326263 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326020 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 20 21:47:14.326263 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326012 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.326263 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326021 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.326918 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326900 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 20 21:47:14.327048 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326900 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jjmbf\"" Apr 20 21:47:14.327048 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.326942 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 20 21:47:14.327226 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.327192 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.329206 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329190 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 20 21:47:14.329463 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329446 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.329507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329476 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.329547 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329530 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bblwp\"" Apr 20 21:47:14.329639 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329609 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-run-netns\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.329686 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329655 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.329686 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329672 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-env-overrides\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.329773 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329692 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e473a6df-1529-4ac3-a1ae-0403db7ceafb-iptables-alerter-script\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.329773 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329743 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e473a6df-1529-4ac3-a1ae-0403db7ceafb-host-slash\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.329970 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329778 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-cni-bin\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.329970 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329859 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-etc-selinux\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.329970 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329887 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-systemd-units\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.329970 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329909 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-slash\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.329970 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329932 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce4147f9-483f-4069-8c46-f85f87f617a4-ovn-node-metrics-cert\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329971 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-ovnkube-script-lib\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.329999 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfskd\" (UniqueName: \"kubernetes.io/projected/ce4147f9-483f-4069-8c46-f85f87f617a4-kube-api-access-dfskd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330040 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-registration-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330070 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-var-lib-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9w45\" (UniqueName: \"kubernetes.io/projected/21cfad4e-0887-4313-affc-bf692b73daad-kube-api-access-w9w45\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.330147 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330134 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-kubelet\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330159 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-systemd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330185 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-log-socket\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330208 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhtkn\" (UniqueName: \"kubernetes.io/projected/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-kube-api-access-xhtkn\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330235 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hwzh\" (UniqueName: \"kubernetes.io/projected/e473a6df-1529-4ac3-a1ae-0403db7ceafb-kube-api-access-8hwzh\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330258 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330298 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ba13249-2427-41e8-98f4-ad5adbb97a2d-konnectivity-ca\") pod \"konnectivity-agent-thbph\" (UID: \"3ba13249-2427-41e8-98f4-ad5adbb97a2d\") " pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330334 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-device-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-sys-fs\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330410 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.330455 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330443 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330482 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-ovn\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330510 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-ovnkube-config\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330526 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ba13249-2427-41e8-98f4-ad5adbb97a2d-agent-certs\") pod \"konnectivity-agent-thbph\" (UID: \"3ba13249-2427-41e8-98f4-ad5adbb97a2d\") " pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330540 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-socket-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330562 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-etc-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330591 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-node-log\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.330968 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.330649 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-cni-netd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.332233 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.332182 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.332374 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.332352 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.332447 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.332395 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 20 21:47:14.332537 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.332519 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j7kzm\"" Apr 20 21:47:14.333012 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.332996 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:14.333096 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.333068 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:14.333381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.333363 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 20 21:47:14.335304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.335287 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.336463 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.336444 2576 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 20 21:47:14.337557 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.337542 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.337786 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.337643 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.337786 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.337709 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.337786 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.337758 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6jqw\"" Apr 20 21:47:14.339756 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.339737 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.339887 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.339873 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.340723 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.340253 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.340723 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.340399 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sdzrt\"" Apr 20 21:47:14.342609 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.342512 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4nmcg\"" Apr 20 21:47:14.342952 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.342829 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 20 21:47:14.342952 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.342848 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 20 21:47:14.342952 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.342894 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.343124 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.343047 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 20 21:47:14.345082 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.345062 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 20 21:47:14.345212 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.345189 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 20 21:47:14.346304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.346288 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-twf4d\"" Apr 20 21:47:14.355780 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.355756 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7dj7q" Apr 20 21:47:14.364806 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.364777 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7dj7q" Apr 20 21:47:14.427115 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.427092 2576 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 20 21:47:14.431324 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431299 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-etc-kubernetes\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.431324 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431327 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-cnibin\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.431515 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431343 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-os-release\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.431515 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431360 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-cni-binary-copy\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.431515 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431375 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7c4a314-11cd-4566-9a22-993779800d6c-serviceca\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.431515 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-cnibin\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.431515 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431481 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-multus-certs\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.431731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431517 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysctl-conf\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.431731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431552 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.431731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431606 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-env-overrides\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.431731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431606 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.431731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431699 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-modprobe-d\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.431731 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431717 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3ceb4e7-5b8a-49fe-a752-ae369638e624-tmp\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431736 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-etc-selinux\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431751 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysconfig\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431765 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-run\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431780 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-host\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431802 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-systemd-units\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431813 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-etc-selinux\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431825 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-slash\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431852 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce4147f9-483f-4069-8c46-f85f87f617a4-ovn-node-metrics-cert\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431861 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-systemd-units\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431875 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-ovnkube-script-lib\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431918 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-slash\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5j74\" (UniqueName: \"kubernetes.io/projected/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-kube-api-access-b5j74\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.432003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.431995 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-cni-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432023 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-os-release\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432046 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-socket-dir-parent\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432069 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnx6b\" (UniqueName: \"kubernetes.io/projected/cd13167d-30d3-485e-a738-4a54ad946027-kube-api-access-pnx6b\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432093 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysctl-d\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432111 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-var-lib-kubelet\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432130 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xhtkn\" (UniqueName: \"kubernetes.io/projected/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-kube-api-access-xhtkn\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432142 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-env-overrides\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432149 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-tmp-dir\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432192 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-conf-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432219 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-system-cni-dir\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432245 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd13167d-30d3-485e-a738-4a54ad946027-cni-binary-copy\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432273 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-cni-bin\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432303 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432328 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-sys-fs\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432353 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-ovn\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432377 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-ovnkube-config\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.432597 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432387 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432408 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ba13249-2427-41e8-98f4-ad5adbb97a2d-agent-certs\") pod \"konnectivity-agent-thbph\" (UID: \"3ba13249-2427-41e8-98f4-ad5adbb97a2d\") " pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432407 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-ovnkube-script-lib\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432448 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-ovn\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432430 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-sys-fs\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432479 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-socket-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432527 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-k8s-cni-cncf-io\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432557 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432561 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-socket-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432571 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432593 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-systemd\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432699 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-etc-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432713 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-node-log\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432727 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-cni-netd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432748 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-system-cni-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432772 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-kubelet\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.433376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432776 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-etc-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432792 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-node-log\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432796 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-hostroot\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432844 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-cni-netd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432856 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce4147f9-483f-4069-8c46-f85f87f617a4-ovnkube-config\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432884 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-lib-modules\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432904 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-tuned\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432922 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-run-netns\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-netns\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432955 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e473a6df-1529-4ac3-a1ae-0403db7ceafb-iptables-alerter-script\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432979 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e473a6df-1529-4ac3-a1ae-0403db7ceafb-host-slash\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432995 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-cni-bin\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-cni-multus\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433059 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e473a6df-1529-4ac3-a1ae-0403db7ceafb-host-slash\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433068 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-cni-bin\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433078 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dfskd\" (UniqueName: \"kubernetes.io/projected/ce4147f9-483f-4069-8c46-f85f87f617a4-kube-api-access-dfskd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.434177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433101 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-registration-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.432980 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-run-netns\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433133 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-var-lib-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433178 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-registration-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433217 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-var-lib-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433263 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-kubernetes\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433278 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zckgc\" (UniqueName: \"kubernetes.io/projected/a3ceb4e7-5b8a-49fe-a752-ae369638e624-kube-api-access-zckgc\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433322 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9w45\" (UniqueName: \"kubernetes.io/projected/21cfad4e-0887-4313-affc-bf692b73daad-kube-api-access-w9w45\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433352 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-kubelet\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433375 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-systemd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433399 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-log-socket\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd13167d-30d3-485e-a738-4a54ad946027-multus-daemon-config\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433429 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-systemd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433440 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e473a6df-1529-4ac3-a1ae-0403db7ceafb-iptables-alerter-script\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433455 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8hwzh\" (UniqueName: \"kubernetes.io/projected/e473a6df-1529-4ac3-a1ae-0403db7ceafb-kube-api-access-8hwzh\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433473 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-log-socket\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433483 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ba13249-2427-41e8-98f4-ad5adbb97a2d-konnectivity-ca\") pod \"konnectivity-agent-thbph\" (UID: \"3ba13249-2427-41e8-98f4-ad5adbb97a2d\") " pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433507 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433512 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-host-kubelet\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433533 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-device-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433572 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433589 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-hosts-file\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433617 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98bvh\" (UniqueName: \"kubernetes.io/projected/1e86e08c-b3e5-4b48-8155-c3b112031e05-kube-api-access-98bvh\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433622 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-device-dir\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433671 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-sys\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433707 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433751 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433776 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7c4a314-11cd-4566-9a22-993779800d6c-host\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433800 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl88f\" (UniqueName: \"kubernetes.io/projected/b7c4a314-11cd-4566-9a22-993779800d6c-kube-api-access-tl88f\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.433860 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433899 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce4147f9-483f-4069-8c46-f85f87f617a4-run-openvswitch\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.433956 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:47:14.933912841 +0000 UTC m=+2.101233575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.433987 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3ba13249-2427-41e8-98f4-ad5adbb97a2d-konnectivity-ca\") pod \"konnectivity-agent-thbph\" (UID: \"3ba13249-2427-41e8-98f4-ad5adbb97a2d\") " pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.435734 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.435560 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce4147f9-483f-4069-8c46-f85f87f617a4-ovn-node-metrics-cert\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.436269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.435669 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3ba13249-2427-41e8-98f4-ad5adbb97a2d-agent-certs\") pod \"konnectivity-agent-thbph\" (UID: \"3ba13249-2427-41e8-98f4-ad5adbb97a2d\") " pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.443792 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.443763 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9w45\" (UniqueName: \"kubernetes.io/projected/21cfad4e-0887-4313-affc-bf692b73daad-kube-api-access-w9w45\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.445446 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.445413 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhtkn\" (UniqueName: \"kubernetes.io/projected/d79d3b04-e2b2-4cbf-9f27-1641f15bd825-kube-api-access-xhtkn\") pod \"aws-ebs-csi-driver-node-gznxc\" (UID: \"d79d3b04-e2b2-4cbf-9f27-1641f15bd825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.446966 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.446534 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hwzh\" (UniqueName: \"kubernetes.io/projected/e473a6df-1529-4ac3-a1ae-0403db7ceafb-kube-api-access-8hwzh\") pod \"iptables-alerter-j45q2\" (UID: \"e473a6df-1529-4ac3-a1ae-0403db7ceafb\") " pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.446966 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.446829 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfskd\" (UniqueName: \"kubernetes.io/projected/ce4147f9-483f-4069-8c46-f85f87f617a4-kube-api-access-dfskd\") pod \"ovnkube-node-vf7zb\" (UID: \"ce4147f9-483f-4069-8c46-f85f87f617a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.473485 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.473282 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3588c42b05d40fa47c543e3dbe98316.slice/crio-bad007809dfe1c559e9e8ba6e2abe2581cc18287069bb1a196b55f1198442eaa WatchSource:0}: Error finding container bad007809dfe1c559e9e8ba6e2abe2581cc18287069bb1a196b55f1198442eaa: Status 404 returned error can't find the container with id bad007809dfe1c559e9e8ba6e2abe2581cc18287069bb1a196b55f1198442eaa Apr 20 21:47:14.474042 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.474008 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba8a1e795b921c317949461306d2685.slice/crio-8b819de5496801911c08247613501b41969da6ffff915f182ff50ccb6e8cb350 WatchSource:0}: Error finding container 8b819de5496801911c08247613501b41969da6ffff915f182ff50ccb6e8cb350: Status 404 returned error can't find the container with id 8b819de5496801911c08247613501b41969da6ffff915f182ff50ccb6e8cb350 Apr 20 21:47:14.477839 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.477817 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 20 21:47:14.534872 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534828 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7c4a314-11cd-4566-9a22-993779800d6c-host\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.534872 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534870 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tl88f\" (UniqueName: \"kubernetes.io/projected/b7c4a314-11cd-4566-9a22-993779800d6c-kube-api-access-tl88f\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534892 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-etc-kubernetes\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534918 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-cnibin\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534940 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-os-release\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534948 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b7c4a314-11cd-4566-9a22-993779800d6c-host\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534960 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-cni-binary-copy\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534952 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-etc-kubernetes\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534981 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7c4a314-11cd-4566-9a22-993779800d6c-serviceca\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.534983 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-cnibin\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535003 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-cnibin\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535044 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-os-release\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535060 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-cnibin\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535048 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-multus-certs\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535098 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysctl-conf\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535127 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-modprobe-d\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535099 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-multus-certs\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535150 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3ceb4e7-5b8a-49fe-a752-ae369638e624-tmp\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535192 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysconfig\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535220 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-run\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535263 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-host\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535277 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-modprobe-d\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535286 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysconfig\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535302 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b5j74\" (UniqueName: \"kubernetes.io/projected/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-kube-api-access-b5j74\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535324 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-cni-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535330 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-host\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535341 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-os-release\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535358 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-socket-dir-parent\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535383 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pnx6b\" (UniqueName: \"kubernetes.io/projected/cd13167d-30d3-485e-a738-4a54ad946027-kube-api-access-pnx6b\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535398 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-run\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535407 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysctl-d\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535416 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-cni-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535358 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysctl-conf\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.535788 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535434 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-var-lib-kubelet\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-os-release\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535469 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-socket-dir-parent\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535461 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-tmp-dir\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535482 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b7c4a314-11cd-4566-9a22-993779800d6c-serviceca\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535503 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-conf-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535517 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-var-lib-kubelet\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535529 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-system-cni-dir\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535554 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-multus-conf-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535564 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd13167d-30d3-485e-a738-4a54ad946027-cni-binary-copy\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535570 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-sysctl-d\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535589 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-cni-bin\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535589 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-cni-binary-copy\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535589 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-system-cni-dir\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535616 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-k8s-cni-cncf-io\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535652 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535673 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-k8s-cni-cncf-io\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.536596 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535680 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-cni-bin\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535711 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535741 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-systemd\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535772 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-system-cni-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535816 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-kubelet\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535826 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e86e08c-b3e5-4b48-8155-c3b112031e05-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535842 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-hostroot\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535856 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-tmp-dir\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535867 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-lib-modules\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535872 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-kubelet\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535874 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-systemd\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535911 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-tuned\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535929 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-hostroot\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535932 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-system-cni-dir\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.535984 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-netns\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536017 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-lib-modules\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536038 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-run-netns\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536046 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-cni-multus\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.537349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536074 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd13167d-30d3-485e-a738-4a54ad946027-host-var-lib-cni-multus\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536078 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536111 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536140 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-kubernetes\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536145 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd13167d-30d3-485e-a738-4a54ad946027-cni-binary-copy\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536165 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zckgc\" (UniqueName: \"kubernetes.io/projected/a3ceb4e7-5b8a-49fe-a752-ae369638e624-kube-api-access-zckgc\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536216 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-kubernetes\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536249 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd13167d-30d3-485e-a738-4a54ad946027-multus-daemon-config\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536280 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-hosts-file\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536285 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536306 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-98bvh\" (UniqueName: \"kubernetes.io/projected/1e86e08c-b3e5-4b48-8155-c3b112031e05-kube-api-access-98bvh\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536360 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-hosts-file\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536396 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-sys\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536478 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a3ceb4e7-5b8a-49fe-a752-ae369638e624-sys\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536565 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e86e08c-b3e5-4b48-8155-c3b112031e05-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.536797 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd13167d-30d3-485e-a738-4a54ad946027-multus-daemon-config\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.538201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.538006 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a3ceb4e7-5b8a-49fe-a752-ae369638e624-etc-tuned\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.538648 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.538239 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a3ceb4e7-5b8a-49fe-a752-ae369638e624-tmp\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.545246 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.545213 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:14.545246 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.545241 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:14.545450 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.545255 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:14.545450 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.545333 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:47:15.045312947 +0000 UTC m=+2.212633664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:14.545795 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.545771 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl88f\" (UniqueName: \"kubernetes.io/projected/b7c4a314-11cd-4566-9a22-993779800d6c-kube-api-access-tl88f\") pod \"node-ca-fhmlz\" (UID: \"b7c4a314-11cd-4566-9a22-993779800d6c\") " pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.546249 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.546234 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnx6b\" (UniqueName: \"kubernetes.io/projected/cd13167d-30d3-485e-a738-4a54ad946027-kube-api-access-pnx6b\") pod \"multus-4c5hz\" (UID: \"cd13167d-30d3-485e-a738-4a54ad946027\") " pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.546602 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.546587 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5j74\" (UniqueName: \"kubernetes.io/projected/e3c32b6a-2f6a-4754-b26a-70179ed0c9eb-kube-api-access-b5j74\") pod \"node-resolver-7t457\" (UID: \"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb\") " pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.547024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.547002 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98bvh\" (UniqueName: \"kubernetes.io/projected/1e86e08c-b3e5-4b48-8155-c3b112031e05-kube-api-access-98bvh\") pod \"multus-additional-cni-plugins-cm52p\" (UID: \"1e86e08c-b3e5-4b48-8155-c3b112031e05\") " pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.547131 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.547113 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zckgc\" (UniqueName: \"kubernetes.io/projected/a3ceb4e7-5b8a-49fe-a752-ae369638e624-kube-api-access-zckgc\") pod \"tuned-dclhd\" (UID: \"a3ceb4e7-5b8a-49fe-a752-ae369638e624\") " pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.653715 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.653665 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j45q2" Apr 20 21:47:14.660823 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.660794 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode473a6df_1529_4ac3_a1ae_0403db7ceafb.slice/crio-dee5175aa87029074c99045a2dd559212b93f43cbdfb8c82b79943e975f338bb WatchSource:0}: Error finding container dee5175aa87029074c99045a2dd559212b93f43cbdfb8c82b79943e975f338bb: Status 404 returned error can't find the container with id dee5175aa87029074c99045a2dd559212b93f43cbdfb8c82b79943e975f338bb Apr 20 21:47:14.667211 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.667189 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:14.673741 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.673711 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4147f9_483f_4069_8c46_f85f87f617a4.slice/crio-c6c08ece1d2b490336adb959e81ec2381a95dbad324f41d84bcb023f15a83a92 WatchSource:0}: Error finding container c6c08ece1d2b490336adb959e81ec2381a95dbad324f41d84bcb023f15a83a92: Status 404 returned error can't find the container with id c6c08ece1d2b490336adb959e81ec2381a95dbad324f41d84bcb023f15a83a92 Apr 20 21:47:14.684297 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.684272 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:14.690459 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.690424 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba13249_2427_41e8_98f4_ad5adbb97a2d.slice/crio-4e413fb8f47ff49835d72d1ed607929f877260f35b30c56bb64949fccb40646e WatchSource:0}: Error finding container 4e413fb8f47ff49835d72d1ed607929f877260f35b30c56bb64949fccb40646e: Status 404 returned error can't find the container with id 4e413fb8f47ff49835d72d1ed607929f877260f35b30c56bb64949fccb40646e Apr 20 21:47:14.698540 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.698497 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" Apr 20 21:47:14.706043 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.706016 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd79d3b04_e2b2_4cbf_9f27_1641f15bd825.slice/crio-5c1bec378bd3f2548f24e54b6773d505e0701032e4155eb0dee9a18f24244d80 WatchSource:0}: Error finding container 5c1bec378bd3f2548f24e54b6773d505e0701032e4155eb0dee9a18f24244d80: Status 404 returned error can't find the container with id 5c1bec378bd3f2548f24e54b6773d505e0701032e4155eb0dee9a18f24244d80 Apr 20 21:47:14.721601 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.721571 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4c5hz" Apr 20 21:47:14.729162 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.729135 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd13167d_30d3_485e_a738_4a54ad946027.slice/crio-fac770afd784a932af67267626d6ddd877ef3fe8021854d332eb6cc19ddfae93 WatchSource:0}: Error finding container fac770afd784a932af67267626d6ddd877ef3fe8021854d332eb6cc19ddfae93: Status 404 returned error can't find the container with id fac770afd784a932af67267626d6ddd877ef3fe8021854d332eb6cc19ddfae93 Apr 20 21:47:14.734428 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.734408 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dclhd" Apr 20 21:47:14.740415 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.740391 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ceb4e7_5b8a_49fe_a752_ae369638e624.slice/crio-7593dd1c9a88dd27417ed28712136caedeae1897598206e6ba0387b853330ab0 WatchSource:0}: Error finding container 7593dd1c9a88dd27417ed28712136caedeae1897598206e6ba0387b853330ab0: Status 404 returned error can't find the container with id 7593dd1c9a88dd27417ed28712136caedeae1897598206e6ba0387b853330ab0 Apr 20 21:47:14.755278 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.755247 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7t457" Apr 20 21:47:14.761363 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.761328 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c32b6a_2f6a_4754_b26a_70179ed0c9eb.slice/crio-a574fba9bf51b66ca7f0a8259029819a2b8aa988458c51c46ab99efe0279a8b8 WatchSource:0}: Error finding container a574fba9bf51b66ca7f0a8259029819a2b8aa988458c51c46ab99efe0279a8b8: Status 404 returned error can't find the container with id a574fba9bf51b66ca7f0a8259029819a2b8aa988458c51c46ab99efe0279a8b8 Apr 20 21:47:14.772795 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.772763 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fhmlz" Apr 20 21:47:14.778471 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.778445 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cm52p" Apr 20 21:47:14.779151 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.779132 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c4a314_11cd_4566_9a22_993779800d6c.slice/crio-2f92c5ddca61cec948b16fd56366d6a76e75a889d75ac306cbced7875bf520e6 WatchSource:0}: Error finding container 2f92c5ddca61cec948b16fd56366d6a76e75a889d75ac306cbced7875bf520e6: Status 404 returned error can't find the container with id 2f92c5ddca61cec948b16fd56366d6a76e75a889d75ac306cbced7875bf520e6 Apr 20 21:47:14.784831 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:47:14.784807 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e86e08c_b3e5_4b48_8155_c3b112031e05.slice/crio-fd89c4a24139932a3d6fbfdf7ac0b14b229e559132d95f79ce24251883cbdd34 WatchSource:0}: Error finding container fd89c4a24139932a3d6fbfdf7ac0b14b229e559132d95f79ce24251883cbdd34: Status 404 returned error can't find the container with id fd89c4a24139932a3d6fbfdf7ac0b14b229e559132d95f79ce24251883cbdd34 Apr 20 21:47:14.940125 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:14.940090 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:14.940325 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.940229 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:14.940391 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:14.940327 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:47:15.94027228 +0000 UTC m=+3.107592999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:15.146245 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.146175 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:15.146472 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.146350 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:15.146472 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.146368 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:15.146472 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.146381 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:15.146472 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.146445 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:47:16.1464202 +0000 UTC m=+3.313740928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:15.333648 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.333600 2576 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:15.353880 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.353849 2576 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:15.366414 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.366372 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-19 21:42:14 +0000 UTC" deadline="2027-10-26 19:25:40.09753097 +0000 UTC" Apr 20 21:47:15.366414 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.366411 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13293h38m24.731124473s" Apr 20 21:47:15.465725 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.465578 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:15.465889 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.465723 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:15.512446 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.512330 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerStarted","Data":"fd89c4a24139932a3d6fbfdf7ac0b14b229e559132d95f79ce24251883cbdd34"} Apr 20 21:47:15.525423 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.525098 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fhmlz" event={"ID":"b7c4a314-11cd-4566-9a22-993779800d6c","Type":"ContainerStarted","Data":"2f92c5ddca61cec948b16fd56366d6a76e75a889d75ac306cbced7875bf520e6"} Apr 20 21:47:15.538644 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.538591 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7t457" event={"ID":"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb","Type":"ContainerStarted","Data":"a574fba9bf51b66ca7f0a8259029819a2b8aa988458c51c46ab99efe0279a8b8"} Apr 20 21:47:15.551730 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.551421 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" event={"ID":"d79d3b04-e2b2-4cbf-9f27-1641f15bd825","Type":"ContainerStarted","Data":"5c1bec378bd3f2548f24e54b6773d505e0701032e4155eb0dee9a18f24244d80"} Apr 20 21:47:15.563337 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.562998 2576 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 20 21:47:15.569370 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.569064 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thbph" event={"ID":"3ba13249-2427-41e8-98f4-ad5adbb97a2d","Type":"ContainerStarted","Data":"4e413fb8f47ff49835d72d1ed607929f877260f35b30c56bb64949fccb40646e"} Apr 20 21:47:15.589385 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.589338 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"c6c08ece1d2b490336adb959e81ec2381a95dbad324f41d84bcb023f15a83a92"} Apr 20 21:47:15.609243 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.609202 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" event={"ID":"eba8a1e795b921c317949461306d2685","Type":"ContainerStarted","Data":"8b819de5496801911c08247613501b41969da6ffff915f182ff50ccb6e8cb350"} Apr 20 21:47:15.631954 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.631913 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dclhd" event={"ID":"a3ceb4e7-5b8a-49fe-a752-ae369638e624","Type":"ContainerStarted","Data":"7593dd1c9a88dd27417ed28712136caedeae1897598206e6ba0387b853330ab0"} Apr 20 21:47:15.648205 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.648164 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4c5hz" event={"ID":"cd13167d-30d3-485e-a738-4a54ad946027","Type":"ContainerStarted","Data":"fac770afd784a932af67267626d6ddd877ef3fe8021854d332eb6cc19ddfae93"} Apr 20 21:47:15.668889 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.667244 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j45q2" event={"ID":"e473a6df-1529-4ac3-a1ae-0403db7ceafb","Type":"ContainerStarted","Data":"dee5175aa87029074c99045a2dd559212b93f43cbdfb8c82b79943e975f338bb"} Apr 20 21:47:15.679830 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.678512 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" event={"ID":"f3588c42b05d40fa47c543e3dbe98316","Type":"ContainerStarted","Data":"bad007809dfe1c559e9e8ba6e2abe2581cc18287069bb1a196b55f1198442eaa"} Apr 20 21:47:15.956234 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:15.956168 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:15.956427 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.956415 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:15.956499 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:15.956483 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:47:17.956462601 +0000 UTC m=+5.123783322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:16.157648 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:16.157582 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:16.157814 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:16.157756 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:16.157814 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:16.157780 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:16.157814 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:16.157792 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:16.157984 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:16.157853 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:47:18.157834563 +0000 UTC m=+5.325155299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:16.367578 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:16.367536 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-19 21:42:14 +0000 UTC" deadline="2028-01-11 04:26:24.879475002 +0000 UTC" Apr 20 21:47:16.368024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:16.367580 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15126h39m8.511899349s" Apr 20 21:47:16.466253 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:16.466218 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:16.466445 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:16.466362 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:17.469514 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:17.468730 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:17.469514 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:17.468854 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:17.974617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:17.974018 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:17.974617 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:17.974197 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:17.974617 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:17.974267 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:47:21.974246352 +0000 UTC m=+9.141567067 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:18.176857 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:18.176812 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:18.177054 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:18.176993 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:18.177054 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:18.177023 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:18.177054 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:18.177037 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:18.177221 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:18.177107 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:47:22.177085854 +0000 UTC m=+9.344406594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:18.465570 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:18.465534 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:18.465760 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:18.465696 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:19.465716 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:19.465681 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:19.466160 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:19.465833 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:20.466357 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:20.466318 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:20.466815 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:20.466483 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:21.465837 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:21.465807 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:21.466013 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:21.465922 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:22.010681 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:22.010623 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:22.011136 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.010752 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:22.011136 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.010836 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:47:30.010814807 +0000 UTC m=+17.178135587 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:22.212877 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:22.212835 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:22.213093 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.213070 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:22.213159 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.213102 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:22.213159 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.213116 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:22.213254 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.213184 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:47:30.213163357 +0000 UTC m=+17.380484105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:22.465774 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:22.465738 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:22.465969 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:22.465885 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:23.467009 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:23.466979 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:23.467461 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:23.467075 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:24.466158 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:24.466119 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:24.466419 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:24.466249 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:25.465736 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:25.465695 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:25.466161 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:25.465833 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:26.466199 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:26.466158 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:26.466659 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:26.466303 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:27.465648 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:27.465609 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:27.465818 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:27.465749 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:28.466241 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:28.466206 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:28.466681 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:28.466348 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:29.466236 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:29.466195 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:29.466405 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:29.466337 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:30.072299 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:30.072249 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:30.072518 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.072384 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:30.072518 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.072457 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:47:46.072440857 +0000 UTC m=+33.239761576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:30.273984 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:30.273941 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:30.274146 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.274124 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:30.274146 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.274147 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:30.274266 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.274159 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:30.274266 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.274228 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:47:46.274210131 +0000 UTC m=+33.441530856 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:30.465562 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:30.465524 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:30.465744 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:30.465673 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:31.466236 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:31.466198 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:31.466641 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:31.466323 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:32.466402 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:32.466370 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:32.466690 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:32.466487 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:33.468064 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.466618 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:33.468064 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:33.467011 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:33.717961 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.717934 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:47:33.718281 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718257 2576 generic.go:358] "Generic (PLEG): container finished" podID="ce4147f9-483f-4069-8c46-f85f87f617a4" containerID="ceadab6276960d148bfd79ce2e06b48c25e7ca5e8cd049a0a216d5a1eae48e1d" exitCode=1 Apr 20 21:47:33.718371 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718327 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"6c8d2457acd98054e43cf9d2a92e056d33dddcbbef36b81722943db019e3fde0"} Apr 20 21:47:33.718371 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718362 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"6fd253505b2ba2cf017dc66ec44319bfeafd8a5aca4350dfcd58c5b524ef46b1"} Apr 20 21:47:33.718456 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718375 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"2a4c6cbdb19522fd017a5ff4a5f99fd100da12280c384d2115b74d725719b22c"} Apr 20 21:47:33.718456 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718387 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"5ff0b6b581e88a2146f26452549f3125c57e9c32e43f2c86ee8244cde7893fa8"} Apr 20 21:47:33.718456 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718397 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerDied","Data":"ceadab6276960d148bfd79ce2e06b48c25e7ca5e8cd049a0a216d5a1eae48e1d"} Apr 20 21:47:33.718456 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.718407 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"67a256a4b67503f2c3db4ba214a0d4f3d8d449f63df7cf98801a747bc7ae90da"} Apr 20 21:47:33.719470 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.719449 2576 generic.go:358] "Generic (PLEG): container finished" podID="eba8a1e795b921c317949461306d2685" containerID="b1f175d79707e390d2fe4b0835e7303e8980fea27908827330cfe5f961776410" exitCode=0 Apr 20 21:47:33.719540 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.719512 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" event={"ID":"eba8a1e795b921c317949461306d2685","Type":"ContainerDied","Data":"b1f175d79707e390d2fe4b0835e7303e8980fea27908827330cfe5f961776410"} Apr 20 21:47:33.720773 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.720746 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dclhd" event={"ID":"a3ceb4e7-5b8a-49fe-a752-ae369638e624","Type":"ContainerStarted","Data":"b664435f84f178c7f839da19c47919c9572ec1d4e55dbe320c4245810987c4d6"} Apr 20 21:47:33.722011 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.721989 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4c5hz" event={"ID":"cd13167d-30d3-485e-a738-4a54ad946027","Type":"ContainerStarted","Data":"c1fdff781dd307fc2884e1007e287af31fe26e95216b49ea72618907cc19d6e2"} Apr 20 21:47:33.723499 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.723475 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" event={"ID":"f3588c42b05d40fa47c543e3dbe98316","Type":"ContainerStarted","Data":"6f50013b6fd3842163745935cc86981e38c0fd0f27abba31641f16eb33aa49c2"} Apr 20 21:47:33.724717 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.724694 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerStarted","Data":"b57f2c9d675e4634fd3d1c01e3be5338c17e9175e1476636a7427c8448e532d8"} Apr 20 21:47:33.730110 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.730084 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fhmlz" event={"ID":"b7c4a314-11cd-4566-9a22-993779800d6c","Type":"ContainerStarted","Data":"db65bde2a741f3d7dd8ef3ad56cba97453afb876dda863ad4e6ce8011f056908"} Apr 20 21:47:33.731243 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.731213 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7t457" event={"ID":"e3c32b6a-2f6a-4754-b26a-70179ed0c9eb","Type":"ContainerStarted","Data":"1ab6cd225eeb68a178023665b8f59375ec6c7df07f7ce9a1092d9c4d3b5a6dc4"} Apr 20 21:47:33.732585 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.732566 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" event={"ID":"d79d3b04-e2b2-4cbf-9f27-1641f15bd825","Type":"ContainerStarted","Data":"3499677141bd940f1566d06a14a2279f51e60a23fc2800d12ab3ecd5477fff93"} Apr 20 21:47:33.733909 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.733884 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thbph" event={"ID":"3ba13249-2427-41e8-98f4-ad5adbb97a2d","Type":"ContainerStarted","Data":"46fb4c4c86c0c5df4064247d51e7c4deb7162e071f98df70b345017bb0c00479"} Apr 20 21:47:33.765860 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.765796 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dclhd" podStartSLOduration=2.812081034 podStartE2EDuration="20.765776342s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.741987515 +0000 UTC m=+1.909308230" lastFinishedPulling="2026-04-20 21:47:32.695682818 +0000 UTC m=+19.863003538" observedRunningTime="2026-04-20 21:47:33.745545331 +0000 UTC m=+20.912866080" watchObservedRunningTime="2026-04-20 21:47:33.765776342 +0000 UTC m=+20.933097079" Apr 20 21:47:33.776058 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.776012 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-102.ec2.internal" podStartSLOduration=20.775993682 podStartE2EDuration="20.775993682s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 21:47:33.775605734 +0000 UTC m=+20.942926469" watchObservedRunningTime="2026-04-20 21:47:33.775993682 +0000 UTC m=+20.943314427" Apr 20 21:47:33.786674 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.786599 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-7t457" podStartSLOduration=2.904638636 podStartE2EDuration="20.786578622s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.762976605 +0000 UTC m=+1.930297319" lastFinishedPulling="2026-04-20 21:47:32.644916576 +0000 UTC m=+19.812237305" observedRunningTime="2026-04-20 21:47:33.786174151 +0000 UTC m=+20.953494888" watchObservedRunningTime="2026-04-20 21:47:33.786578622 +0000 UTC m=+20.953899360" Apr 20 21:47:33.798096 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.798041 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fhmlz" podStartSLOduration=2.913835076 podStartE2EDuration="20.798025526s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.781410676 +0000 UTC m=+1.948731389" lastFinishedPulling="2026-04-20 21:47:32.665601112 +0000 UTC m=+19.832921839" observedRunningTime="2026-04-20 21:47:33.797547117 +0000 UTC m=+20.964867855" watchObservedRunningTime="2026-04-20 21:47:33.798025526 +0000 UTC m=+20.965346265" Apr 20 21:47:33.811219 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.811177 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4c5hz" podStartSLOduration=2.5413143639999998 podStartE2EDuration="20.811164154s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.730834992 +0000 UTC m=+1.898155718" lastFinishedPulling="2026-04-20 21:47:33.000684771 +0000 UTC m=+20.168005508" observedRunningTime="2026-04-20 21:47:33.810769644 +0000 UTC m=+20.978090381" watchObservedRunningTime="2026-04-20 21:47:33.811164154 +0000 UTC m=+20.978484891" Apr 20 21:47:33.827029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:33.826983 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-thbph" podStartSLOduration=2.8736557190000003 podStartE2EDuration="20.826966463s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.691986056 +0000 UTC m=+1.859306770" lastFinishedPulling="2026-04-20 21:47:32.645296794 +0000 UTC m=+19.812617514" observedRunningTime="2026-04-20 21:47:33.826855197 +0000 UTC m=+20.994175933" watchObservedRunningTime="2026-04-20 21:47:33.826966463 +0000 UTC m=+20.994287201" Apr 20 21:47:34.465667 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.465645 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:34.465787 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:34.465743 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:34.487830 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.487675 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 20 21:47:34.737050 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.737005 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j45q2" event={"ID":"e473a6df-1529-4ac3-a1ae-0403db7ceafb","Type":"ContainerStarted","Data":"898c169f140ef469786eb6014416c2be8abe33028fb33c44bd28da90d700a654"} Apr 20 21:47:34.738380 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.738349 2576 generic.go:358] "Generic (PLEG): container finished" podID="1e86e08c-b3e5-4b48-8155-c3b112031e05" containerID="b57f2c9d675e4634fd3d1c01e3be5338c17e9175e1476636a7427c8448e532d8" exitCode=0 Apr 20 21:47:34.738502 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.738423 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerDied","Data":"b57f2c9d675e4634fd3d1c01e3be5338c17e9175e1476636a7427c8448e532d8"} Apr 20 21:47:34.740230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.740206 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" event={"ID":"d79d3b04-e2b2-4cbf-9f27-1641f15bd825","Type":"ContainerStarted","Data":"fab6c55fb6c33c8580fcd5c35be32e569054cc63b6f55ebb250e56c6f67f9a4a"} Apr 20 21:47:34.741882 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.741861 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" event={"ID":"eba8a1e795b921c317949461306d2685","Type":"ContainerStarted","Data":"db2f100a1fb083a6c6c1907a9f3d2763239e7d5fe92c2688306db8140d03918a"} Apr 20 21:47:34.749345 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.749305 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j45q2" podStartSLOduration=3.719176505 podStartE2EDuration="21.749285604s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.662442306 +0000 UTC m=+1.829763020" lastFinishedPulling="2026-04-20 21:47:32.692551405 +0000 UTC m=+19.859872119" observedRunningTime="2026-04-20 21:47:34.748889603 +0000 UTC m=+21.916210340" watchObservedRunningTime="2026-04-20 21:47:34.749285604 +0000 UTC m=+21.916606339" Apr 20 21:47:34.760391 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:34.760343 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-102.ec2.internal" podStartSLOduration=21.760327894 podStartE2EDuration="21.760327894s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 21:47:34.759962155 +0000 UTC m=+21.927282892" watchObservedRunningTime="2026-04-20 21:47:34.760327894 +0000 UTC m=+21.927648629" Apr 20 21:47:35.401692 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.401557 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-20T21:47:34.487822904Z","UUID":"6465bcff-b0cf-442f-be21-bd8230cb2835","Handler":null,"Name":"","Endpoint":""} Apr 20 21:47:35.404168 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.404066 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 20 21:47:35.404168 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.404100 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 20 21:47:35.466274 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.466251 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:35.466395 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:35.466361 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:35.748243 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.748158 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" event={"ID":"d79d3b04-e2b2-4cbf-9f27-1641f15bd825","Type":"ContainerStarted","Data":"2553ce7c7e057eeb143b64627edf4d246ba0bc2f938dba63f5c9b5d518c27d2f"} Apr 20 21:47:35.751299 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.751275 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:47:35.751755 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.751707 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"4bd75d6d14f08ca103b9980f6c094e91f9a4223e6e37f31bdeccf50146b697d9"} Apr 20 21:47:35.763768 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:35.763712 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gznxc" podStartSLOduration=2.276664849 podStartE2EDuration="22.763693407s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.707415332 +0000 UTC m=+1.874736046" lastFinishedPulling="2026-04-20 21:47:35.194443883 +0000 UTC m=+22.361764604" observedRunningTime="2026-04-20 21:47:35.762736639 +0000 UTC m=+22.930057375" watchObservedRunningTime="2026-04-20 21:47:35.763693407 +0000 UTC m=+22.931014144" Apr 20 21:47:36.466457 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:36.466420 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:36.466676 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:36.466571 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:37.466307 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:37.466268 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:37.467051 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:37.466380 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:38.465466 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:38.465424 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:38.465672 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:38.465552 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:38.678347 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:38.678316 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:38.678920 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:38.678899 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:38.757981 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:38.757795 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:38.758406 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:38.758390 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-thbph" Apr 20 21:47:39.465865 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.465834 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:39.466034 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:39.465935 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:39.761337 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.761311 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:47:39.761778 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.761658 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"dd5d4f584b8e75ab85f9e98691bc1694577dbb1bd2f12594b03c4fd0835984c2"} Apr 20 21:47:39.762033 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.762009 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:39.762033 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.762038 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:39.762219 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.762187 2576 scope.go:117] "RemoveContainer" containerID="ceadab6276960d148bfd79ce2e06b48c25e7ca5e8cd049a0a216d5a1eae48e1d" Apr 20 21:47:39.763406 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.763384 2576 generic.go:358] "Generic (PLEG): container finished" podID="1e86e08c-b3e5-4b48-8155-c3b112031e05" containerID="22ba97354e7578071868f56d5a8e1c76bd477c1ec7abfd3463b5c4493e19c209" exitCode=0 Apr 20 21:47:39.763512 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.763482 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerDied","Data":"22ba97354e7578071868f56d5a8e1c76bd477c1ec7abfd3463b5c4493e19c209"} Apr 20 21:47:39.779079 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:39.779046 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:40.465617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.465584 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:40.465781 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:40.465707 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:40.737188 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.736962 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnz85"] Apr 20 21:47:40.737329 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.737233 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:40.737329 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:40.737317 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:40.739794 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.739766 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7b48f"] Apr 20 21:47:40.768184 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.768152 2576 generic.go:358] "Generic (PLEG): container finished" podID="1e86e08c-b3e5-4b48-8155-c3b112031e05" containerID="78281ba9011f75c24061e8600059af4affab166f0b280b925bce7b0d900361e4" exitCode=0 Apr 20 21:47:40.768710 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.768241 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerDied","Data":"78281ba9011f75c24061e8600059af4affab166f0b280b925bce7b0d900361e4"} Apr 20 21:47:40.771987 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.771958 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:47:40.772454 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.772425 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" event={"ID":"ce4147f9-483f-4069-8c46-f85f87f617a4","Type":"ContainerStarted","Data":"7d715e3f0bbbcb5d40619f0473b8bee669ac21e296a893fc95efeecfad44b420"} Apr 20 21:47:40.772803 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.772761 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:40.772915 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:40.772875 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:40.772915 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.772909 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:40.788609 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.788583 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:47:40.817710 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:40.817662 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" podStartSLOduration=9.759541049 podStartE2EDuration="27.81764502s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.675293375 +0000 UTC m=+1.842614091" lastFinishedPulling="2026-04-20 21:47:32.733397341 +0000 UTC m=+19.900718062" observedRunningTime="2026-04-20 21:47:40.815766747 +0000 UTC m=+27.983087482" watchObservedRunningTime="2026-04-20 21:47:40.81764502 +0000 UTC m=+27.984965750" Apr 20 21:47:41.776606 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:41.776576 2576 generic.go:358] "Generic (PLEG): container finished" podID="1e86e08c-b3e5-4b48-8155-c3b112031e05" containerID="f975e9496e66f7eb552845fc6fe16dd92934e54f241aa3f125bd52f5a92b3cdc" exitCode=0 Apr 20 21:47:41.777144 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:41.776670 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerDied","Data":"f975e9496e66f7eb552845fc6fe16dd92934e54f241aa3f125bd52f5a92b3cdc"} Apr 20 21:47:42.466166 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:42.466131 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:42.466344 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:42.466131 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:42.466344 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:42.466264 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:42.466344 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:42.466335 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:44.466014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:44.465977 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:44.466560 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:44.465977 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:44.466560 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:44.466106 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:47:44.466560 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:44.466225 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wnz85" podUID="da17023a-5926-4acb-a4c5-265519c40a9e" Apr 20 21:47:46.091619 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.091383 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:46.092061 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.091548 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:46.092061 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.091768 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:48:18.091750528 +0000 UTC m=+65.259071242 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 20 21:47:46.160982 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.160951 2576 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-102.ec2.internal" event="NodeReady" Apr 20 21:47:46.161152 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.161104 2576 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 20 21:47:46.205895 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.205864 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p2pgl"] Apr 20 21:47:46.230506 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.230472 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6gm99"] Apr 20 21:47:46.230705 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.230681 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.233440 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.233413 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 20 21:47:46.233587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.233440 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 20 21:47:46.233587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.233421 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4j5s\"" Apr 20 21:47:46.248683 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.248656 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p2pgl"] Apr 20 21:47:46.248683 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.248681 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6gm99"] Apr 20 21:47:46.248953 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.248749 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.252799 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.252738 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 20 21:47:46.252799 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.252790 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 20 21:47:46.253140 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.253123 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bqmdj\"" Apr 20 21:47:46.253203 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.253132 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 20 21:47:46.293934 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.293900 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:46.294133 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.294081 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 20 21:47:46.294133 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.294106 2576 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 20 21:47:46.294133 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.294119 2576 projected.go:194] Error preparing data for projected volume kube-api-access-8lwr7 for pod openshift-network-diagnostics/network-check-target-wnz85: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:46.294290 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.294189 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7 podName:da17023a-5926-4acb-a4c5-265519c40a9e nodeName:}" failed. No retries permitted until 2026-04-20 21:48:18.294175177 +0000 UTC m=+65.461495891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8lwr7" (UniqueName: "kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7") pod "network-check-target-wnz85" (UID: "da17023a-5926-4acb-a4c5-265519c40a9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 20 21:47:46.394621 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.394572 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gxb9\" (UniqueName: \"kubernetes.io/projected/cce82b4c-1d52-4715-a259-01734afb2a79-kube-api-access-5gxb9\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.394621 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.394615 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d733792-4f92-4b9a-8436-4427d99740cd-config-volume\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.394906 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.394658 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d733792-4f92-4b9a-8436-4427d99740cd-tmp-dir\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.394906 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.394683 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9flq\" (UniqueName: \"kubernetes.io/projected/2d733792-4f92-4b9a-8436-4427d99740cd-kube-api-access-k9flq\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.394906 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.394742 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.394906 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.394805 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.466016 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.465967 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:47:46.466232 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.465967 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:47:46.468850 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.468827 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 20 21:47:46.469006 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.468851 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wzfs\"" Apr 20 21:47:46.469006 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.468909 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hbjh6\"" Apr 20 21:47:46.469006 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.468916 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 20 21:47:46.469006 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.468948 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 20 21:47:46.495523 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495495 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d733792-4f92-4b9a-8436-4427d99740cd-tmp-dir\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.495523 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495528 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-k9flq\" (UniqueName: \"kubernetes.io/projected/2d733792-4f92-4b9a-8436-4427d99740cd-kube-api-access-k9flq\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.495768 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495549 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.495768 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.495664 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:47:46.495768 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.495719 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:47:46.995700318 +0000 UTC m=+34.163021037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:47:46.495768 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495738 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.495962 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495809 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5gxb9\" (UniqueName: \"kubernetes.io/projected/cce82b4c-1d52-4715-a259-01734afb2a79-kube-api-access-5gxb9\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.495962 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495831 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d733792-4f92-4b9a-8436-4427d99740cd-config-volume\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.495962 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.495844 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d733792-4f92-4b9a-8436-4427d99740cd-tmp-dir\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.495962 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.495924 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:47:46.496146 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.495972 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:47:46.995955051 +0000 UTC m=+34.163275780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:47:46.496294 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.496275 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2d733792-4f92-4b9a-8436-4427d99740cd-config-volume\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.509300 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.509268 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gxb9\" (UniqueName: \"kubernetes.io/projected/cce82b4c-1d52-4715-a259-01734afb2a79-kube-api-access-5gxb9\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.509469 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.509266 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9flq\" (UniqueName: \"kubernetes.io/projected/2d733792-4f92-4b9a-8436-4427d99740cd-kube-api-access-k9flq\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.999233 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.999189 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:46.999482 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.999352 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:47:46.999482 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:46.999379 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:46.999482 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.999431 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:47:47.999410362 +0000 UTC m=+35.166731086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:47:46.999482 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.999481 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:47:46.999732 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:46.999536 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:47:47.999523764 +0000 UTC m=+35.166844477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:47:47.790994 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:47.790964 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerStarted","Data":"88ef0b73a095166eb58ccebe135c19d9d47147e371cc4ad0e1d8df3d458ea62d"} Apr 20 21:47:48.007402 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:48.007364 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:48.007600 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:48.007467 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:48.007600 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:48.007527 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:47:48.007600 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:48.007583 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:47:48.007761 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:48.007594 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:47:50.007576831 +0000 UTC m=+37.174897551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:47:48.007761 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:48.007674 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:47:50.007660088 +0000 UTC m=+37.174980806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:47:48.795786 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:48.795751 2576 generic.go:358] "Generic (PLEG): container finished" podID="1e86e08c-b3e5-4b48-8155-c3b112031e05" containerID="88ef0b73a095166eb58ccebe135c19d9d47147e371cc4ad0e1d8df3d458ea62d" exitCode=0 Apr 20 21:47:48.796218 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:48.795829 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerDied","Data":"88ef0b73a095166eb58ccebe135c19d9d47147e371cc4ad0e1d8df3d458ea62d"} Apr 20 21:47:49.800604 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:49.800573 2576 generic.go:358] "Generic (PLEG): container finished" podID="1e86e08c-b3e5-4b48-8155-c3b112031e05" containerID="9121965c79f8ed5738ba9a61b7cc185b2bb16bf6726fd33aaf2369650accc2d9" exitCode=0 Apr 20 21:47:49.801067 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:49.800612 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerDied","Data":"9121965c79f8ed5738ba9a61b7cc185b2bb16bf6726fd33aaf2369650accc2d9"} Apr 20 21:47:50.022294 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:50.022259 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:50.022451 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:50.022305 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:50.022451 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:50.022403 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:47:50.022525 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:50.022459 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:47:54.022444734 +0000 UTC m=+41.189765448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:47:50.022525 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:50.022403 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:47:50.022593 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:50.022534 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:47:54.022521954 +0000 UTC m=+41.189842667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:47:50.805339 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:50.805305 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm52p" event={"ID":"1e86e08c-b3e5-4b48-8155-c3b112031e05","Type":"ContainerStarted","Data":"33cf5cc5955ac098823cf6f3179bb8edf4b6b73b0e0061984b8adfa51de9e808"} Apr 20 21:47:50.828339 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:50.828290 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cm52p" podStartSLOduration=4.998162501 podStartE2EDuration="37.828273559s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:47:14.786221188 +0000 UTC m=+1.953541905" lastFinishedPulling="2026-04-20 21:47:47.616332235 +0000 UTC m=+34.783652963" observedRunningTime="2026-04-20 21:47:50.826384061 +0000 UTC m=+37.993704797" watchObservedRunningTime="2026-04-20 21:47:50.828273559 +0000 UTC m=+37.995594349" Apr 20 21:47:54.048514 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:54.048477 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:47:54.048921 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:47:54.048549 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:47:54.048921 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:54.048624 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:47:54.048921 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:54.048711 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:48:02.048692559 +0000 UTC m=+49.216013273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:47:54.048921 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:54.048649 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:47:54.048921 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:47:54.048743 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:48:02.048736717 +0000 UTC m=+49.216057430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:48:02.103317 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:02.103263 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:48:02.103831 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:02.103333 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:48:02.103831 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:02.103395 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:48:02.103831 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:02.103446 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:48:02.103831 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:02.103458 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:48:18.103443216 +0000 UTC m=+65.270763931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:48:02.103831 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:02.103499 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:48:18.103483304 +0000 UTC m=+65.270804039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:48:12.789426 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:12.789394 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vf7zb" Apr 20 21:48:18.113735 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.113691 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:48:18.114141 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.113750 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:48:18.114141 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.113787 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:48:18.114141 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:18.113872 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:48:18.114141 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:18.113951 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:48:50.113935151 +0000 UTC m=+97.281255865 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:48:18.114141 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:18.113878 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:48:18.114141 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:18.114049 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:48:50.114029986 +0000 UTC m=+97.281350705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:48:18.116557 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.116543 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 20 21:48:18.123934 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:18.123910 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 20 21:48:18.124071 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:18.123996 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:49:22.123974146 +0000 UTC m=+129.291294860 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : secret "metrics-daemon-secret" not found Apr 20 21:48:18.315080 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.315037 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:48:18.318118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.318100 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 20 21:48:18.328429 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.328407 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 20 21:48:18.340522 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.340491 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lwr7\" (UniqueName: \"kubernetes.io/projected/da17023a-5926-4acb-a4c5-265519c40a9e-kube-api-access-8lwr7\") pod \"network-check-target-wnz85\" (UID: \"da17023a-5926-4acb-a4c5-265519c40a9e\") " pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:48:18.581044 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.581010 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hbjh6\"" Apr 20 21:48:18.589225 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.589201 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:48:18.779921 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.779892 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wnz85"] Apr 20 21:48:18.784061 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:48:18.784029 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda17023a_5926_4acb_a4c5_265519c40a9e.slice/crio-4f8520462e4e946a3f2d4067b6ddc20232dfb30ecdd14b676ff84d82a8c0879b WatchSource:0}: Error finding container 4f8520462e4e946a3f2d4067b6ddc20232dfb30ecdd14b676ff84d82a8c0879b: Status 404 returned error can't find the container with id 4f8520462e4e946a3f2d4067b6ddc20232dfb30ecdd14b676ff84d82a8c0879b Apr 20 21:48:18.862136 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:18.862052 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnz85" event={"ID":"da17023a-5926-4acb-a4c5-265519c40a9e","Type":"ContainerStarted","Data":"4f8520462e4e946a3f2d4067b6ddc20232dfb30ecdd14b676ff84d82a8c0879b"} Apr 20 21:48:21.870202 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:21.870163 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wnz85" event={"ID":"da17023a-5926-4acb-a4c5-265519c40a9e","Type":"ContainerStarted","Data":"bee724eaed3257168db201d937a675a74a3ae5ef3ce699131fec75f7bc5c47a6"} Apr 20 21:48:21.870562 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:21.870412 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:48:21.885318 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:21.885263 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wnz85" podStartSLOduration=66.183364085 podStartE2EDuration="1m8.885249272s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:48:18.786335132 +0000 UTC m=+65.953655846" lastFinishedPulling="2026-04-20 21:48:21.488220312 +0000 UTC m=+68.655541033" observedRunningTime="2026-04-20 21:48:21.884072512 +0000 UTC m=+69.051393249" watchObservedRunningTime="2026-04-20 21:48:21.885249272 +0000 UTC m=+69.052570007" Apr 20 21:48:50.136942 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:50.136882 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:48:50.136942 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:50.136957 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:48:50.137667 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:50.137069 2576 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 20 21:48:50.137667 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:50.137131 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert podName:cce82b4c-1d52-4715-a259-01734afb2a79 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:54.137116676 +0000 UTC m=+161.304437396 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert") pod "ingress-canary-6gm99" (UID: "cce82b4c-1d52-4715-a259-01734afb2a79") : secret "canary-serving-cert" not found Apr 20 21:48:50.137667 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:50.137073 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 20 21:48:50.137667 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:48:50.137217 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls podName:2d733792-4f92-4b9a-8436-4427d99740cd nodeName:}" failed. No retries permitted until 2026-04-20 21:49:54.137198389 +0000 UTC m=+161.304519110 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls") pod "dns-default-p2pgl" (UID: "2d733792-4f92-4b9a-8436-4427d99740cd") : secret "dns-default-metrics-tls" not found Apr 20 21:48:52.875352 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:48:52.875320 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wnz85" Apr 20 21:49:22.168229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:22.168176 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:49:22.168796 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:22.168330 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 20 21:49:22.168796 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:22.168398 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs podName:21cfad4e-0887-4313-affc-bf692b73daad nodeName:}" failed. No retries permitted until 2026-04-20 21:51:24.168379782 +0000 UTC m=+251.335700496 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs") pod "network-metrics-daemon-7b48f" (UID: "21cfad4e-0887-4313-affc-bf692b73daad") : secret "metrics-daemon-secret" not found Apr 20 21:49:31.066973 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.066939 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-585dfdc468-bjk5j"] Apr 20 21:49:31.069292 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.069275 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.072200 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.072174 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Apr 20 21:49:31.072200 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.072200 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.072400 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.072230 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.072400 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.072267 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Apr 20 21:49:31.072497 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.072485 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-76kcz\"" Apr 20 21:49:31.078756 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.078718 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Apr 20 21:49:31.079740 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.079717 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-bjk5j"] Apr 20 21:49:31.165333 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.165299 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r"] Apr 20 21:49:31.171336 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.171305 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-wwr45"] Apr 20 21:49:31.171503 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.171467 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" Apr 20 21:49:31.173859 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.173833 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.174261 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.174241 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.174771 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.174754 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-pvpw9\"" Apr 20 21:49:31.175091 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.175076 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.177394 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.177377 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.177529 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.177411 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Apr 20 21:49:31.177529 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.177473 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Apr 20 21:49:31.178762 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.178393 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-n8zjd\"" Apr 20 21:49:31.178762 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.178710 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r"] Apr 20 21:49:31.178875 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.178782 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.182728 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.182704 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-wwr45"] Apr 20 21:49:31.184003 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.183977 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Apr 20 21:49:31.232314 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.232275 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05c5b9c5-70a5-43e5-9bb2-864e21b29416-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.232503 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.232332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05c5b9c5-70a5-43e5-9bb2-864e21b29416-tmp\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.232503 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.232361 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05c5b9c5-70a5-43e5-9bb2-864e21b29416-serving-cert\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.232503 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.232393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/05c5b9c5-70a5-43e5-9bb2-864e21b29416-snapshots\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.232503 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.232413 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfxw7\" (UniqueName: \"kubernetes.io/projected/05c5b9c5-70a5-43e5-9bb2-864e21b29416-kube-api-access-qfxw7\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.232503 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.232478 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05c5b9c5-70a5-43e5-9bb2-864e21b29416-service-ca-bundle\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.270825 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.270789 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn"] Apr 20 21:49:31.273330 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.273305 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp"] Apr 20 21:49:31.273479 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.273450 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.275980 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.275958 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn"] Apr 20 21:49:31.276118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.276099 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" Apr 20 21:49:31.276256 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.276237 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ljrq4\"" Apr 20 21:49:31.277696 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.277676 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Apr 20 21:49:31.277791 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.277717 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.278365 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.278352 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt"] Apr 20 21:49:31.278501 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.278488 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.279077 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.279058 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Apr 20 21:49:31.279199 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.279099 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.279199 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.279114 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gmjhx\"" Apr 20 21:49:31.280924 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.280906 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.281178 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.281160 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Apr 20 21:49:31.281267 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.281174 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.281267 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.281196 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Apr 20 21:49:31.281506 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.281492 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-4xtpc\"" Apr 20 21:49:31.281608 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.281587 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7"] Apr 20 21:49:31.281745 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.281728 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:31.284259 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.284239 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn"] Apr 20 21:49:31.284361 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.284350 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.284437 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.284382 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.284437 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.284391 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.284437 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.284382 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Apr 20 21:49:31.284582 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.284473 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rxk88\"" Apr 20 21:49:31.286998 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.286979 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Apr 20 21:49:31.287346 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.287324 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:49:31.287438 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.287333 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Apr 20 21:49:31.287518 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.287497 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7"] Apr 20 21:49:31.287832 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.287790 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Apr 20 21:49:31.288024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.288004 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-4k6gg\"" Apr 20 21:49:31.288370 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.288350 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn"] Apr 20 21:49:31.295963 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.295937 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp"] Apr 20 21:49:31.300701 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.300673 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt"] Apr 20 21:49:31.333376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333291 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86a52490-01a5-444d-bf45-653d11ab3fc6-config\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.333376 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333357 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05c5b9c5-70a5-43e5-9bb2-864e21b29416-service-ca-bundle\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.333589 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86a52490-01a5-444d-bf45-653d11ab3fc6-trusted-ca\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.333589 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333410 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz4vx\" (UniqueName: \"kubernetes.io/projected/34d1dcb2-71b0-4b76-b5b9-9195fb6f4494-kube-api-access-hz4vx\") pod \"volume-data-source-validator-7c6cbb6c87-lgv4r\" (UID: \"34d1dcb2-71b0-4b76-b5b9-9195fb6f4494\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" Apr 20 21:49:31.333589 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333442 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05c5b9c5-70a5-43e5-9bb2-864e21b29416-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.333589 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333491 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05c5b9c5-70a5-43e5-9bb2-864e21b29416-tmp\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.333589 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333513 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05c5b9c5-70a5-43e5-9bb2-864e21b29416-serving-cert\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.333589 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333541 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86a52490-01a5-444d-bf45-653d11ab3fc6-serving-cert\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.333872 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333661 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xp2g\" (UniqueName: \"kubernetes.io/projected/86a52490-01a5-444d-bf45-653d11ab3fc6-kube-api-access-8xp2g\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.333872 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333708 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/05c5b9c5-70a5-43e5-9bb2-864e21b29416-snapshots\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.333872 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333776 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qfxw7\" (UniqueName: \"kubernetes.io/projected/05c5b9c5-70a5-43e5-9bb2-864e21b29416-kube-api-access-qfxw7\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.334020 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.333950 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05c5b9c5-70a5-43e5-9bb2-864e21b29416-tmp\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.334071 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.334043 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05c5b9c5-70a5-43e5-9bb2-864e21b29416-service-ca-bundle\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.334229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.334209 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/05c5b9c5-70a5-43e5-9bb2-864e21b29416-snapshots\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.334273 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.334253 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05c5b9c5-70a5-43e5-9bb2-864e21b29416-trusted-ca-bundle\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.336053 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.336031 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05c5b9c5-70a5-43e5-9bb2-864e21b29416-serving-cert\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.344352 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.344324 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfxw7\" (UniqueName: \"kubernetes.io/projected/05c5b9c5-70a5-43e5-9bb2-864e21b29416-kube-api-access-qfxw7\") pod \"insights-operator-585dfdc468-bjk5j\" (UID: \"05c5b9c5-70a5-43e5-9bb2-864e21b29416\") " pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.377382 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.377346 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" Apr 20 21:49:31.434998 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.434964 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7561c1f-02bf-450a-9019-4a0e67eb82a3-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.434998 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435004 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzvlk\" (UniqueName: \"kubernetes.io/projected/b7561c1f-02bf-450a-9019-4a0e67eb82a3-kube-api-access-mzvlk\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnfzs\" (UniqueName: \"kubernetes.io/projected/c468f16b-183a-4a08-92af-8e224d82525c-kube-api-access-rnfzs\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435041 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435088 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86a52490-01a5-444d-bf45-653d11ab3fc6-serving-cert\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435137 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8xp2g\" (UniqueName: \"kubernetes.io/projected/86a52490-01a5-444d-bf45-653d11ab3fc6-kube-api-access-8xp2g\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435154 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7ztg\" (UniqueName: \"kubernetes.io/projected/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-kube-api-access-v7ztg\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435184 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wv2\" (UniqueName: \"kubernetes.io/projected/66600852-86ff-417d-8fb2-ae9f8c624a67-kube-api-access-r9wv2\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435203 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86a52490-01a5-444d-bf45-653d11ab3fc6-config\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.435229 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435217 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c468f16b-183a-4a08-92af-8e224d82525c-config\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.435720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435243 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx9ch\" (UniqueName: \"kubernetes.io/projected/f994290e-016b-430c-9937-4bd4a04a725e-kube-api-access-kx9ch\") pod \"network-check-source-8894fc9bd-l59hp\" (UID: \"f994290e-016b-430c-9937-4bd4a04a725e\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" Apr 20 21:49:31.435720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435267 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c468f16b-183a-4a08-92af-8e224d82525c-serving-cert\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.435720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435319 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:31.435720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435356 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86a52490-01a5-444d-bf45-653d11ab3fc6-trusted-ca\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.435720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435384 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hz4vx\" (UniqueName: \"kubernetes.io/projected/34d1dcb2-71b0-4b76-b5b9-9195fb6f4494-kube-api-access-hz4vx\") pod \"volume-data-source-validator-7c6cbb6c87-lgv4r\" (UID: \"34d1dcb2-71b0-4b76-b5b9-9195fb6f4494\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" Apr 20 21:49:31.435720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.435412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.436989 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.436915 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86a52490-01a5-444d-bf45-653d11ab3fc6-config\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.436989 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.436978 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86a52490-01a5-444d-bf45-653d11ab3fc6-trusted-ca\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.438664 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.438596 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86a52490-01a5-444d-bf45-653d11ab3fc6-serving-cert\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.444803 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.444771 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xp2g\" (UniqueName: \"kubernetes.io/projected/86a52490-01a5-444d-bf45-653d11ab3fc6-kube-api-access-8xp2g\") pod \"console-operator-9d4b6777b-wwr45\" (UID: \"86a52490-01a5-444d-bf45-653d11ab3fc6\") " pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.444903 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.444851 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz4vx\" (UniqueName: \"kubernetes.io/projected/34d1dcb2-71b0-4b76-b5b9-9195fb6f4494-kube-api-access-hz4vx\") pod \"volume-data-source-validator-7c6cbb6c87-lgv4r\" (UID: \"34d1dcb2-71b0-4b76-b5b9-9195fb6f4494\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" Apr 20 21:49:31.481901 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.481869 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" Apr 20 21:49:31.488369 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.488339 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:31.490759 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.490412 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-585dfdc468-bjk5j"] Apr 20 21:49:31.496041 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:31.496005 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c5b9c5_70a5_43e5_9bb2_864e21b29416.slice/crio-290fc16c04cd4826ceeb0d9ea4224111cd06c3724ce03ddf2549a40af938f3e6 WatchSource:0}: Error finding container 290fc16c04cd4826ceeb0d9ea4224111cd06c3724ce03ddf2549a40af938f3e6: Status 404 returned error can't find the container with id 290fc16c04cd4826ceeb0d9ea4224111cd06c3724ce03ddf2549a40af938f3e6 Apr 20 21:49:31.536729 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536700 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wv2\" (UniqueName: \"kubernetes.io/projected/66600852-86ff-417d-8fb2-ae9f8c624a67-kube-api-access-r9wv2\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:31.536889 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536740 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c468f16b-183a-4a08-92af-8e224d82525c-config\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.536889 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536772 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kx9ch\" (UniqueName: \"kubernetes.io/projected/f994290e-016b-430c-9937-4bd4a04a725e-kube-api-access-kx9ch\") pod \"network-check-source-8894fc9bd-l59hp\" (UID: \"f994290e-016b-430c-9937-4bd4a04a725e\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" Apr 20 21:49:31.536889 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536801 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c468f16b-183a-4a08-92af-8e224d82525c-serving-cert\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.536889 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536848 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:31.536889 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536879 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.537148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536911 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7561c1f-02bf-450a-9019-4a0e67eb82a3-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.537148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536940 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mzvlk\" (UniqueName: \"kubernetes.io/projected/b7561c1f-02bf-450a-9019-4a0e67eb82a3-kube-api-access-mzvlk\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.537148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.536974 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rnfzs\" (UniqueName: \"kubernetes.io/projected/c468f16b-183a-4a08-92af-8e224d82525c-kube-api-access-rnfzs\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.537148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.537002 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.537148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.537065 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.537148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.537097 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7ztg\" (UniqueName: \"kubernetes.io/projected/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-kube-api-access-v7ztg\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.537876 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:31.537621 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:31.537876 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:31.537755 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls podName:b7561c1f-02bf-450a-9019-4a0e67eb82a3 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:32.037734363 +0000 UTC m=+139.205055091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-hqzpn" (UID: "b7561c1f-02bf-450a-9019-4a0e67eb82a3") : secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:31.537876 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.537808 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-config\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.538209 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:31.537953 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 20 21:49:31.538209 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:31.538028 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls podName:66600852-86ff-417d-8fb2-ae9f8c624a67 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:32.038008292 +0000 UTC m=+139.205329023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-2jwbt" (UID: "66600852-86ff-417d-8fb2-ae9f8c624a67") : secret "samples-operator-tls" not found Apr 20 21:49:31.538320 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.538226 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c468f16b-183a-4a08-92af-8e224d82525c-config\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.538833 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.538804 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7561c1f-02bf-450a-9019-4a0e67eb82a3-telemetry-config\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.542041 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.541839 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c468f16b-183a-4a08-92af-8e224d82525c-serving-cert\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.542041 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.541939 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-serving-cert\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.549992 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.549929 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7ztg\" (UniqueName: \"kubernetes.io/projected/0aaaf0fe-29e0-4335-bb5d-a7f0551f94da-kube-api-access-v7ztg\") pod \"kube-storage-version-migrator-operator-6769c5d45-sfsfn\" (UID: \"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.550778 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.550737 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx9ch\" (UniqueName: \"kubernetes.io/projected/f994290e-016b-430c-9937-4bd4a04a725e-kube-api-access-kx9ch\") pod \"network-check-source-8894fc9bd-l59hp\" (UID: \"f994290e-016b-430c-9937-4bd4a04a725e\") " pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" Apr 20 21:49:31.551064 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.551039 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnfzs\" (UniqueName: \"kubernetes.io/projected/c468f16b-183a-4a08-92af-8e224d82525c-kube-api-access-rnfzs\") pod \"service-ca-operator-d6fc45fc5-qx6f7\" (UID: \"c468f16b-183a-4a08-92af-8e224d82525c\") " pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.551153 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.551091 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wv2\" (UniqueName: \"kubernetes.io/projected/66600852-86ff-417d-8fb2-ae9f8c624a67-kube-api-access-r9wv2\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:31.552425 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.552380 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzvlk\" (UniqueName: \"kubernetes.io/projected/b7561c1f-02bf-450a-9019-4a0e67eb82a3-kube-api-access-mzvlk\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:31.593068 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.593032 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" Apr 20 21:49:31.598851 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.598817 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" Apr 20 21:49:31.607724 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.607694 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r"] Apr 20 21:49:31.610730 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.610691 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" Apr 20 21:49:31.610864 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:31.610839 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d1dcb2_71b0_4b76_b5b9_9195fb6f4494.slice/crio-706f8e8b7d161f644276532efbfcc3e376504bc014061ca4623203f01e7b1144 WatchSource:0}: Error finding container 706f8e8b7d161f644276532efbfcc3e376504bc014061ca4623203f01e7b1144: Status 404 returned error can't find the container with id 706f8e8b7d161f644276532efbfcc3e376504bc014061ca4623203f01e7b1144 Apr 20 21:49:31.629349 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.629303 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-9d4b6777b-wwr45"] Apr 20 21:49:31.632753 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:31.632698 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86a52490_01a5_444d_bf45_653d11ab3fc6.slice/crio-8f0790053a8c598395926a2c5397cf03b15186fd9ffcc205b45af16f03e8b497 WatchSource:0}: Error finding container 8f0790053a8c598395926a2c5397cf03b15186fd9ffcc205b45af16f03e8b497: Status 404 returned error can't find the container with id 8f0790053a8c598395926a2c5397cf03b15186fd9ffcc205b45af16f03e8b497 Apr 20 21:49:31.736424 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:31.736391 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf994290e_016b_430c_9937_4bd4a04a725e.slice/crio-4d194df03323382472a2b40c457279c6eda3e659a1827580eb0662cf93ece8a1 WatchSource:0}: Error finding container 4d194df03323382472a2b40c457279c6eda3e659a1827580eb0662cf93ece8a1: Status 404 returned error can't find the container with id 4d194df03323382472a2b40c457279c6eda3e659a1827580eb0662cf93ece8a1 Apr 20 21:49:31.738307 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.738252 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp"] Apr 20 21:49:31.751958 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.751924 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn"] Apr 20 21:49:31.758008 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:31.755837 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aaaf0fe_29e0_4335_bb5d_a7f0551f94da.slice/crio-06fac7302de67b476bb6d72dca3f9a407db4f3e9065ebed6643a688124c617d9 WatchSource:0}: Error finding container 06fac7302de67b476bb6d72dca3f9a407db4f3e9065ebed6643a688124c617d9: Status 404 returned error can't find the container with id 06fac7302de67b476bb6d72dca3f9a407db4f3e9065ebed6643a688124c617d9 Apr 20 21:49:31.764905 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:31.764875 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7"] Apr 20 21:49:31.770749 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:31.770722 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc468f16b_183a_4a08_92af_8e224d82525c.slice/crio-a780cd6751ca0ef309785926d6ab4e9e16b76ad529e49ebef7435fd3a56e0b2b WatchSource:0}: Error finding container a780cd6751ca0ef309785926d6ab4e9e16b76ad529e49ebef7435fd3a56e0b2b: Status 404 returned error can't find the container with id a780cd6751ca0ef309785926d6ab4e9e16b76ad529e49ebef7435fd3a56e0b2b Apr 20 21:49:32.001388 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.001349 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" event={"ID":"f994290e-016b-430c-9937-4bd4a04a725e","Type":"ContainerStarted","Data":"8ae276f04b17c356d207314390719b1e62a60b217bee7f09618135ecff40cb81"} Apr 20 21:49:32.001388 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.001387 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" event={"ID":"f994290e-016b-430c-9937-4bd4a04a725e","Type":"ContainerStarted","Data":"4d194df03323382472a2b40c457279c6eda3e659a1827580eb0662cf93ece8a1"} Apr 20 21:49:32.002366 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.002336 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" event={"ID":"86a52490-01a5-444d-bf45-653d11ab3fc6","Type":"ContainerStarted","Data":"8f0790053a8c598395926a2c5397cf03b15186fd9ffcc205b45af16f03e8b497"} Apr 20 21:49:32.003295 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.003270 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" event={"ID":"05c5b9c5-70a5-43e5-9bb2-864e21b29416","Type":"ContainerStarted","Data":"290fc16c04cd4826ceeb0d9ea4224111cd06c3724ce03ddf2549a40af938f3e6"} Apr 20 21:49:32.004143 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.004124 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" event={"ID":"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da","Type":"ContainerStarted","Data":"06fac7302de67b476bb6d72dca3f9a407db4f3e9065ebed6643a688124c617d9"} Apr 20 21:49:32.005118 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.005087 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" event={"ID":"34d1dcb2-71b0-4b76-b5b9-9195fb6f4494","Type":"ContainerStarted","Data":"706f8e8b7d161f644276532efbfcc3e376504bc014061ca4623203f01e7b1144"} Apr 20 21:49:32.006043 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.006025 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" event={"ID":"c468f16b-183a-4a08-92af-8e224d82525c","Type":"ContainerStarted","Data":"a780cd6751ca0ef309785926d6ab4e9e16b76ad529e49ebef7435fd3a56e0b2b"} Apr 20 21:49:32.018026 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.017983 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-8894fc9bd-l59hp" podStartSLOduration=1.017968459 podStartE2EDuration="1.017968459s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 21:49:32.016609407 +0000 UTC m=+139.183930144" watchObservedRunningTime="2026-04-20 21:49:32.017968459 +0000 UTC m=+139.185289189" Apr 20 21:49:32.044565 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.044531 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:32.044730 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:32.044580 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:32.044730 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:32.044705 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 20 21:49:32.044806 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:32.044736 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:32.044806 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:32.044771 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls podName:66600852-86ff-417d-8fb2-ae9f8c624a67 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:33.04475386 +0000 UTC m=+140.212074579 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-2jwbt" (UID: "66600852-86ff-417d-8fb2-ae9f8c624a67") : secret "samples-operator-tls" not found Apr 20 21:49:32.044806 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:32.044785 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls podName:b7561c1f-02bf-450a-9019-4a0e67eb82a3 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:33.044779298 +0000 UTC m=+140.212100012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-hqzpn" (UID: "b7561c1f-02bf-450a-9019-4a0e67eb82a3") : secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:33.053990 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:33.053267 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:33.053990 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:33.053337 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:33.053990 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:33.053483 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:33.053990 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:33.053547 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls podName:b7561c1f-02bf-450a-9019-4a0e67eb82a3 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:35.053528087 +0000 UTC m=+142.220848806 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-hqzpn" (UID: "b7561c1f-02bf-450a-9019-4a0e67eb82a3") : secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:33.053990 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:33.053973 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 20 21:49:33.054565 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:33.054026 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls podName:66600852-86ff-417d-8fb2-ae9f8c624a67 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:35.054011686 +0000 UTC m=+142.221332399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-2jwbt" (UID: "66600852-86ff-417d-8fb2-ae9f8c624a67") : secret "samples-operator-tls" not found Apr 20 21:49:35.071818 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.071776 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:35.072184 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.071842 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:35.072184 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.071930 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 20 21:49:35.072184 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.071974 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:35.072184 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.071996 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls podName:66600852-86ff-417d-8fb2-ae9f8c624a67 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:39.07197918 +0000 UTC m=+146.239299900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-2jwbt" (UID: "66600852-86ff-417d-8fb2-ae9f8c624a67") : secret "samples-operator-tls" not found Apr 20 21:49:35.072184 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.072029 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls podName:b7561c1f-02bf-450a-9019-4a0e67eb82a3 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:39.072015491 +0000 UTC m=+146.239336204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-hqzpn" (UID: "b7561c1f-02bf-450a-9019-4a0e67eb82a3") : secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:35.262574 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.262540 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-vl97c"] Apr 20 21:49:35.266964 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.266936 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.269706 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.269619 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Apr 20 21:49:35.269871 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.269822 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-p6brn\"" Apr 20 21:49:35.269871 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.269836 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Apr 20 21:49:35.272932 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.272829 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.273050 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.273013 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/03b4777a-708e-40d9-892d-37883a45e306-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.275181 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.275153 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-vl97c"] Apr 20 21:49:35.374407 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.374365 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/03b4777a-708e-40d9-892d-37883a45e306-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.374591 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.374442 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.374666 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.374590 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 20 21:49:35.374709 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.374674 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert podName:03b4777a-708e-40d9-892d-37883a45e306 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:35.874658509 +0000 UTC m=+143.041979227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-vl97c" (UID: "03b4777a-708e-40d9-892d-37883a45e306") : secret "networking-console-plugin-cert" not found Apr 20 21:49:35.375166 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.375143 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/03b4777a-708e-40d9-892d-37883a45e306-nginx-conf\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.879809 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:35.879703 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:35.879955 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.879875 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 20 21:49:35.879995 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:35.879957 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert podName:03b4777a-708e-40d9-892d-37883a45e306 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:36.879936038 +0000 UTC m=+144.047256757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-vl97c" (UID: "03b4777a-708e-40d9-892d-37883a45e306") : secret "networking-console-plugin-cert" not found Apr 20 21:49:36.018733 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.018706 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/0.log" Apr 20 21:49:36.018924 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.018745 2576 generic.go:358] "Generic (PLEG): container finished" podID="86a52490-01a5-444d-bf45-653d11ab3fc6" containerID="1b780f268329aee486c976e8fd2f99523bf1044f179cf1082e976827d9b417fe" exitCode=255 Apr 20 21:49:36.018924 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.018781 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" event={"ID":"86a52490-01a5-444d-bf45-653d11ab3fc6","Type":"ContainerDied","Data":"1b780f268329aee486c976e8fd2f99523bf1044f179cf1082e976827d9b417fe"} Apr 20 21:49:36.019298 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.019275 2576 scope.go:117] "RemoveContainer" containerID="1b780f268329aee486c976e8fd2f99523bf1044f179cf1082e976827d9b417fe" Apr 20 21:49:36.020306 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.020275 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" event={"ID":"05c5b9c5-70a5-43e5-9bb2-864e21b29416","Type":"ContainerStarted","Data":"92a52a4b9650bda1783bffa45634615d60981a8155a137628a978686876e36a7"} Apr 20 21:49:36.021695 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.021665 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" event={"ID":"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da","Type":"ContainerStarted","Data":"669d3b8f47a3abf53507b2e3e715e7812fe5420d4766d8ca0c0642e096db42c8"} Apr 20 21:49:36.023129 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.023107 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" event={"ID":"34d1dcb2-71b0-4b76-b5b9-9195fb6f4494","Type":"ContainerStarted","Data":"05d0ad0d112e3e47670fb26e226ce83cc45a91b2fdf7222a95d785815282af6c"} Apr 20 21:49:36.024456 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.024425 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" event={"ID":"c468f16b-183a-4a08-92af-8e224d82525c","Type":"ContainerStarted","Data":"56e7bad4a3c6db55abff97591cf2d0aa706116ee3219ba0af6fb1a169709db1d"} Apr 20 21:49:36.048875 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.048823 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" podStartSLOduration=1.19830611 podStartE2EDuration="5.048808069s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:49:31.772752181 +0000 UTC m=+138.940072895" lastFinishedPulling="2026-04-20 21:49:35.623254125 +0000 UTC m=+142.790574854" observedRunningTime="2026-04-20 21:49:36.047750043 +0000 UTC m=+143.215070781" watchObservedRunningTime="2026-04-20 21:49:36.048808069 +0000 UTC m=+143.216128804" Apr 20 21:49:36.084621 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.081869 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7c6cbb6c87-lgv4r" podStartSLOduration=1.079041521 podStartE2EDuration="5.081849902s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:49:31.613222315 +0000 UTC m=+138.780543053" lastFinishedPulling="2026-04-20 21:49:35.61603072 +0000 UTC m=+142.783351434" observedRunningTime="2026-04-20 21:49:36.066604462 +0000 UTC m=+143.233925199" watchObservedRunningTime="2026-04-20 21:49:36.081849902 +0000 UTC m=+143.249170642" Apr 20 21:49:36.084621 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.082771 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" podStartSLOduration=0.962000029 podStartE2EDuration="5.082761825s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:49:31.49843713 +0000 UTC m=+138.665757843" lastFinishedPulling="2026-04-20 21:49:35.619198911 +0000 UTC m=+142.786519639" observedRunningTime="2026-04-20 21:49:36.081479488 +0000 UTC m=+143.248800224" watchObservedRunningTime="2026-04-20 21:49:36.082761825 +0000 UTC m=+143.250082561" Apr 20 21:49:36.106209 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.106026 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" podStartSLOduration=1.245735022 podStartE2EDuration="5.106004435s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:49:31.759389557 +0000 UTC m=+138.926710270" lastFinishedPulling="2026-04-20 21:49:35.619658955 +0000 UTC m=+142.786979683" observedRunningTime="2026-04-20 21:49:36.103736905 +0000 UTC m=+143.271057642" watchObservedRunningTime="2026-04-20 21:49:36.106004435 +0000 UTC m=+143.273325175" Apr 20 21:49:36.888515 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:36.888419 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:36.888731 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:36.888544 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 20 21:49:36.888731 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:36.888600 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert podName:03b4777a-708e-40d9-892d-37883a45e306 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:38.88858501 +0000 UTC m=+146.055905724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-vl97c" (UID: "03b4777a-708e-40d9-892d-37883a45e306") : secret "networking-console-plugin-cert" not found Apr 20 21:49:37.028841 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:37.028804 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:49:37.029201 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:37.029186 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/0.log" Apr 20 21:49:37.029296 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:37.029222 2576 generic.go:358] "Generic (PLEG): container finished" podID="86a52490-01a5-444d-bf45-653d11ab3fc6" containerID="8e1ca4f8d12d6435311ab27cdc921629ee7edae81e29b0717323c486ace42ea0" exitCode=255 Apr 20 21:49:37.029365 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:37.029336 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" event={"ID":"86a52490-01a5-444d-bf45-653d11ab3fc6","Type":"ContainerDied","Data":"8e1ca4f8d12d6435311ab27cdc921629ee7edae81e29b0717323c486ace42ea0"} Apr 20 21:49:37.029426 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:37.029391 2576 scope.go:117] "RemoveContainer" containerID="1b780f268329aee486c976e8fd2f99523bf1044f179cf1082e976827d9b417fe" Apr 20 21:49:37.029699 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:37.029677 2576 scope.go:117] "RemoveContainer" containerID="8e1ca4f8d12d6435311ab27cdc921629ee7edae81e29b0717323c486ace42ea0" Apr 20 21:49:37.029957 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:37.029925 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-wwr45_openshift-console-operator(86a52490-01a5-444d-bf45-653d11ab3fc6)\"" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" podUID="86a52490-01a5-444d-bf45-653d11ab3fc6" Apr 20 21:49:38.033112 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:38.033082 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:49:38.033509 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:38.033424 2576 scope.go:117] "RemoveContainer" containerID="8e1ca4f8d12d6435311ab27cdc921629ee7edae81e29b0717323c486ace42ea0" Apr 20 21:49:38.033609 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:38.033591 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-wwr45_openshift-console-operator(86a52490-01a5-444d-bf45-653d11ab3fc6)\"" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" podUID="86a52490-01a5-444d-bf45-653d11ab3fc6" Apr 20 21:49:38.332492 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:38.332414 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7t457_e3c32b6a-2f6a-4754-b26a-70179ed0c9eb/dns-node-resolver/0.log" Apr 20 21:49:38.900906 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:38.900852 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:38.901101 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:38.901005 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 20 21:49:38.901101 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:38.901083 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert podName:03b4777a-708e-40d9-892d-37883a45e306 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:42.901067716 +0000 UTC m=+150.068388429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-vl97c" (UID: "03b4777a-708e-40d9-892d-37883a45e306") : secret "networking-console-plugin-cert" not found Apr 20 21:49:39.102430 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.102360 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:39.102887 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.102494 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:39.102887 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:39.102524 2576 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Apr 20 21:49:39.102887 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:39.102590 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls podName:66600852-86ff-417d-8fb2-ae9f8c624a67 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:47.102574539 +0000 UTC m=+154.269895253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls") pod "cluster-samples-operator-6dc5bdb6b4-2jwbt" (UID: "66600852-86ff-417d-8fb2-ae9f8c624a67") : secret "samples-operator-tls" not found Apr 20 21:49:39.102887 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:39.102595 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:39.102887 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:39.102673 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls podName:b7561c1f-02bf-450a-9019-4a0e67eb82a3 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:47.102657366 +0000 UTC m=+154.269978091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-hqzpn" (UID: "b7561c1f-02bf-450a-9019-4a0e67eb82a3") : secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:39.332183 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.332106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fhmlz_b7c4a314-11cd-4566-9a22-993779800d6c/node-ca/0.log" Apr 20 21:49:39.900058 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.900025 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-865cb79987-544cl"] Apr 20 21:49:39.904287 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.904265 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:39.906939 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.906914 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-prwdl\"" Apr 20 21:49:39.907234 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.906958 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Apr 20 21:49:39.908142 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.908121 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Apr 20 21:49:39.908408 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.908388 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Apr 20 21:49:39.908513 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.908391 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Apr 20 21:49:39.910945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:39.910923 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-544cl"] Apr 20 21:49:40.008955 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.008907 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjm6k\" (UniqueName: \"kubernetes.io/projected/b2d85098-9528-466a-b41f-475b989bdf0c-kube-api-access-rjm6k\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.008955 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.008952 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b2d85098-9528-466a-b41f-475b989bdf0c-signing-key\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.009180 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.008970 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b2d85098-9528-466a-b41f-475b989bdf0c-signing-cabundle\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.110050 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.110010 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rjm6k\" (UniqueName: \"kubernetes.io/projected/b2d85098-9528-466a-b41f-475b989bdf0c-kube-api-access-rjm6k\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.110050 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.110053 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b2d85098-9528-466a-b41f-475b989bdf0c-signing-key\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.110464 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.110069 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b2d85098-9528-466a-b41f-475b989bdf0c-signing-cabundle\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.110790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.110771 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b2d85098-9528-466a-b41f-475b989bdf0c-signing-cabundle\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.112604 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.112579 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b2d85098-9528-466a-b41f-475b989bdf0c-signing-key\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.118198 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.118165 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjm6k\" (UniqueName: \"kubernetes.io/projected/b2d85098-9528-466a-b41f-475b989bdf0c-kube-api-access-rjm6k\") pod \"service-ca-865cb79987-544cl\" (UID: \"b2d85098-9528-466a-b41f-475b989bdf0c\") " pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.218497 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.218395 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-865cb79987-544cl" Apr 20 21:49:40.337986 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:40.337948 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-865cb79987-544cl"] Apr 20 21:49:40.341289 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:40.341257 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d85098_9528_466a_b41f_475b989bdf0c.slice/crio-72ab2bd7558f691981e95c4b0a56301c99051ecb92babc5cb23326aa5cb12147 WatchSource:0}: Error finding container 72ab2bd7558f691981e95c4b0a56301c99051ecb92babc5cb23326aa5cb12147: Status 404 returned error can't find the container with id 72ab2bd7558f691981e95c4b0a56301c99051ecb92babc5cb23326aa5cb12147 Apr 20 21:49:41.042220 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:41.042182 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-544cl" event={"ID":"b2d85098-9528-466a-b41f-475b989bdf0c","Type":"ContainerStarted","Data":"fa3c85c0d51474d1c1b4d47b9f78917cdf38dd2f7e97989fcd700320c82ab215"} Apr 20 21:49:41.042220 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:41.042220 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-865cb79987-544cl" event={"ID":"b2d85098-9528-466a-b41f-475b989bdf0c","Type":"ContainerStarted","Data":"72ab2bd7558f691981e95c4b0a56301c99051ecb92babc5cb23326aa5cb12147"} Apr 20 21:49:41.058392 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:41.058343 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-865cb79987-544cl" podStartSLOduration=2.058326982 podStartE2EDuration="2.058326982s" podCreationTimestamp="2026-04-20 21:49:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 21:49:41.058139336 +0000 UTC m=+148.225460072" watchObservedRunningTime="2026-04-20 21:49:41.058326982 +0000 UTC m=+148.225647777" Apr 20 21:49:41.489536 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:41.489499 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:41.489971 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:41.489547 2576 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:41.490025 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:41.490011 2576 scope.go:117] "RemoveContainer" containerID="8e1ca4f8d12d6435311ab27cdc921629ee7edae81e29b0717323c486ace42ea0" Apr 20 21:49:41.490246 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:41.490226 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-9d4b6777b-wwr45_openshift-console-operator(86a52490-01a5-444d-bf45-653d11ab3fc6)\"" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" podUID="86a52490-01a5-444d-bf45-653d11ab3fc6" Apr 20 21:49:42.934294 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:42.934246 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:42.934723 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:42.934387 2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Apr 20 21:49:42.934723 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:42.934457 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert podName:03b4777a-708e-40d9-892d-37883a45e306 nodeName:}" failed. No retries permitted until 2026-04-20 21:49:50.934440216 +0000 UTC m=+158.101760930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert") pod "networking-console-plugin-cb95c66f6-vl97c" (UID: "03b4777a-708e-40d9-892d-37883a45e306") : secret "networking-console-plugin-cert" not found Apr 20 21:49:47.173698 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:47.173661 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:49:47.174155 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:47.173819 2576 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:47.174155 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:47.173836 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:47.174155 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:47.173881 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls podName:b7561c1f-02bf-450a-9019-4a0e67eb82a3 nodeName:}" failed. No retries permitted until 2026-04-20 21:50:03.173865446 +0000 UTC m=+170.341186165 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-75587bd455-hqzpn" (UID: "b7561c1f-02bf-450a-9019-4a0e67eb82a3") : secret "cluster-monitoring-operator-tls" not found Apr 20 21:49:47.176488 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:47.176461 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66600852-86ff-417d-8fb2-ae9f8c624a67-samples-operator-tls\") pod \"cluster-samples-operator-6dc5bdb6b4-2jwbt\" (UID: \"66600852-86ff-417d-8fb2-ae9f8c624a67\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:47.205822 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:47.205788 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" Apr 20 21:49:47.326563 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:47.326377 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt"] Apr 20 21:49:48.062720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:48.062681 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" event={"ID":"66600852-86ff-417d-8fb2-ae9f8c624a67","Type":"ContainerStarted","Data":"04ffadd81dc51f37974318b3a960fa5b523d1eccffd85d85ba83b0fbb8d9c685"} Apr 20 21:49:49.242571 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:49.242527 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-p2pgl" podUID="2d733792-4f92-4b9a-8436-4427d99740cd" Apr 20 21:49:49.259853 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:49.259815 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-6gm99" podUID="cce82b4c-1d52-4715-a259-01734afb2a79" Apr 20 21:49:49.485591 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:49:49.485551 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-7b48f" podUID="21cfad4e-0887-4313-affc-bf692b73daad" Apr 20 21:49:50.067796 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:50.067755 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" event={"ID":"66600852-86ff-417d-8fb2-ae9f8c624a67","Type":"ContainerStarted","Data":"ddf4ad43f6cadcfd286d66668c33c6b7d02e07c20835d2c36e7c1e0ef361b4f8"} Apr 20 21:49:50.067946 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:50.067806 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" event={"ID":"66600852-86ff-417d-8fb2-ae9f8c624a67","Type":"ContainerStarted","Data":"6f81435a55e71f740798e9be7f8ebe39d356e6c5ec22beadc72b312e8389ea0a"} Apr 20 21:49:50.067946 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:50.067764 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p2pgl" Apr 20 21:49:50.067946 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:50.067867 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:49:50.085330 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:50.085284 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6dc5bdb6b4-2jwbt" podStartSLOduration=17.408392254 podStartE2EDuration="19.085270077s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:49:47.383988011 +0000 UTC m=+154.551308734" lastFinishedPulling="2026-04-20 21:49:49.060865806 +0000 UTC m=+156.228186557" observedRunningTime="2026-04-20 21:49:50.083841811 +0000 UTC m=+157.251162548" watchObservedRunningTime="2026-04-20 21:49:50.085270077 +0000 UTC m=+157.252590877" Apr 20 21:49:51.007302 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:51.007247 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:51.009860 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:51.009832 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/03b4777a-708e-40d9-892d-37883a45e306-networking-console-plugin-cert\") pod \"networking-console-plugin-cb95c66f6-vl97c\" (UID: \"03b4777a-708e-40d9-892d-37883a45e306\") " pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:51.177869 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:51.177830 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" Apr 20 21:49:51.296528 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:51.296442 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-cb95c66f6-vl97c"] Apr 20 21:49:51.299201 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:51.299175 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b4777a_708e_40d9_892d_37883a45e306.slice/crio-f9be8d9307d198aa6f42ce338e5bf433fb4b6b1ee683bf2d7b21d1edfb3e320b WatchSource:0}: Error finding container f9be8d9307d198aa6f42ce338e5bf433fb4b6b1ee683bf2d7b21d1edfb3e320b: Status 404 returned error can't find the container with id f9be8d9307d198aa6f42ce338e5bf433fb4b6b1ee683bf2d7b21d1edfb3e320b Apr 20 21:49:52.076346 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:52.076287 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" event={"ID":"03b4777a-708e-40d9-892d-37883a45e306","Type":"ContainerStarted","Data":"f9be8d9307d198aa6f42ce338e5bf433fb4b6b1ee683bf2d7b21d1edfb3e320b"} Apr 20 21:49:52.466761 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:52.466730 2576 scope.go:117] "RemoveContainer" containerID="8e1ca4f8d12d6435311ab27cdc921629ee7edae81e29b0717323c486ace42ea0" Apr 20 21:49:53.080418 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.080380 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" event={"ID":"03b4777a-708e-40d9-892d-37883a45e306","Type":"ContainerStarted","Data":"a64ec56378e43d55d59699a153ea136b67d3b95d20d6b8eb6416e66ca6de6a3f"} Apr 20 21:49:53.081964 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.081945 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:49:53.082134 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.081988 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" event={"ID":"86a52490-01a5-444d-bf45-653d11ab3fc6","Type":"ContainerStarted","Data":"4c3fc694a431007b9a061cf1792f63c39a300f01970f02dfbfa74e537433ea91"} Apr 20 21:49:53.082223 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.082207 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:53.114576 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.114527 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" podStartSLOduration=18.126578982 podStartE2EDuration="22.114511249s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:49:31.634562487 +0000 UTC m=+138.801883201" lastFinishedPulling="2026-04-20 21:49:35.622494754 +0000 UTC m=+142.789815468" observedRunningTime="2026-04-20 21:49:53.113554111 +0000 UTC m=+160.280874848" watchObservedRunningTime="2026-04-20 21:49:53.114511249 +0000 UTC m=+160.281831983" Apr 20 21:49:53.115005 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.114981 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-cb95c66f6-vl97c" podStartSLOduration=17.146266948 podStartE2EDuration="18.114974249s" podCreationTimestamp="2026-04-20 21:49:35 +0000 UTC" firstStartedPulling="2026-04-20 21:49:51.301098298 +0000 UTC m=+158.468419017" lastFinishedPulling="2026-04-20 21:49:52.269805604 +0000 UTC m=+159.437126318" observedRunningTime="2026-04-20 21:49:53.097234669 +0000 UTC m=+160.264555406" watchObservedRunningTime="2026-04-20 21:49:53.114974249 +0000 UTC m=+160.282294985" Apr 20 21:49:53.371355 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:53.371328 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-9d4b6777b-wwr45" Apr 20 21:49:54.149204 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.149152 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:49:54.149655 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.149236 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:49:54.151542 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.151521 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2d733792-4f92-4b9a-8436-4427d99740cd-metrics-tls\") pod \"dns-default-p2pgl\" (UID: \"2d733792-4f92-4b9a-8436-4427d99740cd\") " pod="openshift-dns/dns-default-p2pgl" Apr 20 21:49:54.151678 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.151661 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cce82b4c-1d52-4715-a259-01734afb2a79-cert\") pod \"ingress-canary-6gm99\" (UID: \"cce82b4c-1d52-4715-a259-01734afb2a79\") " pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:49:54.272828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.272776 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s4j5s\"" Apr 20 21:49:54.272828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.272776 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bqmdj\"" Apr 20 21:49:54.280019 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.279990 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6gm99" Apr 20 21:49:54.280172 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.280087 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p2pgl" Apr 20 21:49:54.408956 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.408833 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p2pgl"] Apr 20 21:49:54.413326 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:54.413295 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d733792_4f92_4b9a_8436_4427d99740cd.slice/crio-fd21bebae4ea2b0ad924a9ccc46be15682f62256f0395f2aae908b060d371c38 WatchSource:0}: Error finding container fd21bebae4ea2b0ad924a9ccc46be15682f62256f0395f2aae908b060d371c38: Status 404 returned error can't find the container with id fd21bebae4ea2b0ad924a9ccc46be15682f62256f0395f2aae908b060d371c38 Apr 20 21:49:54.434656 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:54.432785 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6gm99"] Apr 20 21:49:54.438321 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:49:54.438292 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce82b4c_1d52_4715_a259_01734afb2a79.slice/crio-9c0ab9af40e1623f8b553cbc11aee344d94c3393e6bf59f3350a81c426da10c8 WatchSource:0}: Error finding container 9c0ab9af40e1623f8b553cbc11aee344d94c3393e6bf59f3350a81c426da10c8: Status 404 returned error can't find the container with id 9c0ab9af40e1623f8b553cbc11aee344d94c3393e6bf59f3350a81c426da10c8 Apr 20 21:49:55.090646 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:55.090592 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p2pgl" event={"ID":"2d733792-4f92-4b9a-8436-4427d99740cd","Type":"ContainerStarted","Data":"fd21bebae4ea2b0ad924a9ccc46be15682f62256f0395f2aae908b060d371c38"} Apr 20 21:49:55.091965 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:55.091902 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6gm99" event={"ID":"cce82b4c-1d52-4715-a259-01734afb2a79","Type":"ContainerStarted","Data":"9c0ab9af40e1623f8b553cbc11aee344d94c3393e6bf59f3350a81c426da10c8"} Apr 20 21:49:57.098558 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:57.098524 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p2pgl" event={"ID":"2d733792-4f92-4b9a-8436-4427d99740cd","Type":"ContainerStarted","Data":"d62cd0a091b98ce52e9fc34846a08a76cc69cbbfa67d1adbe0022715dbe38f8f"} Apr 20 21:49:57.098558 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:57.098561 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p2pgl" event={"ID":"2d733792-4f92-4b9a-8436-4427d99740cd","Type":"ContainerStarted","Data":"fb51cc2949107f7e56b3accea134aeee9c800935761969b97d8e99ceb0513839"} Apr 20 21:49:57.099064 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:57.098659 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p2pgl" Apr 20 21:49:57.099823 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:57.099798 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6gm99" event={"ID":"cce82b4c-1d52-4715-a259-01734afb2a79","Type":"ContainerStarted","Data":"3501689a98eda5ca26c66b4eabb770bbc768cc65e8e3091a9e8fa801689c530b"} Apr 20 21:49:57.114750 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:57.114708 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-p2pgl" podStartSLOduration=129.304612368 podStartE2EDuration="2m11.114694381s" podCreationTimestamp="2026-04-20 21:47:46 +0000 UTC" firstStartedPulling="2026-04-20 21:49:54.415693326 +0000 UTC m=+161.583014044" lastFinishedPulling="2026-04-20 21:49:56.225775341 +0000 UTC m=+163.393096057" observedRunningTime="2026-04-20 21:49:57.114207575 +0000 UTC m=+164.281528311" watchObservedRunningTime="2026-04-20 21:49:57.114694381 +0000 UTC m=+164.282015117" Apr 20 21:49:57.128398 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:57.128356 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6gm99" podStartSLOduration=129.33934961 podStartE2EDuration="2m11.12834233s" podCreationTimestamp="2026-04-20 21:47:46 +0000 UTC" firstStartedPulling="2026-04-20 21:49:54.440177409 +0000 UTC m=+161.607498126" lastFinishedPulling="2026-04-20 21:49:56.229170132 +0000 UTC m=+163.396490846" observedRunningTime="2026-04-20 21:49:57.127222032 +0000 UTC m=+164.294542767" watchObservedRunningTime="2026-04-20 21:49:57.12834233 +0000 UTC m=+164.295663065" Apr 20 21:49:59.743485 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.743444 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2m6n7"] Apr 20 21:49:59.747111 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.747088 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.750933 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.750906 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-txw8c\"" Apr 20 21:49:59.751073 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.750909 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 20 21:49:59.751073 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.750917 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 20 21:49:59.759972 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.759943 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2m6n7"] Apr 20 21:49:59.875547 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.875515 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c4546fd96-hthl8"] Apr 20 21:49:59.878807 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.878780 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.881809 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.881778 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Apr 20 21:49:59.881960 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.881872 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rxx7z\"" Apr 20 21:49:59.882044 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.881969 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Apr 20 21:49:59.882091 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.882063 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Apr 20 21:49:59.892392 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.892363 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Apr 20 21:49:59.892770 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.892745 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c4546fd96-hthl8"] Apr 20 21:49:59.895655 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.895615 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad6c338e-0e70-428e-9ba3-2c84deef393b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.895760 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.895710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad6c338e-0e70-428e-9ba3-2c84deef393b-data-volume\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.895760 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.895737 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad6c338e-0e70-428e-9ba3-2c84deef393b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.895853 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.895760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tjtk\" (UniqueName: \"kubernetes.io/projected/ad6c338e-0e70-428e-9ba3-2c84deef393b-kube-api-access-8tjtk\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.895853 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.895784 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad6c338e-0e70-428e-9ba3-2c84deef393b-crio-socket\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.996744 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996613 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad6c338e-0e70-428e-9ba3-2c84deef393b-crio-socket\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.996744 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996706 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4b1c201-05e2-42c4-9557-0d84c7354b16-image-registry-private-configuration\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.996982 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996746 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad6c338e-0e70-428e-9ba3-2c84deef393b-crio-socket\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.996982 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tkr9\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-kube-api-access-8tkr9\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.996982 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996840 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4b1c201-05e2-42c4-9557-0d84c7354b16-ca-trust-extracted\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.996982 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996881 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4b1c201-05e2-42c4-9557-0d84c7354b16-installation-pull-secrets\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.996982 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-bound-sa-token\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.997247 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.996998 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c201-05e2-42c4-9557-0d84c7354b16-trusted-ca\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.997247 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997045 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad6c338e-0e70-428e-9ba3-2c84deef393b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.997247 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-registry-tls\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.997247 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997131 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad6c338e-0e70-428e-9ba3-2c84deef393b-data-volume\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.997247 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997163 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad6c338e-0e70-428e-9ba3-2c84deef393b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.997247 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997212 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8tjtk\" (UniqueName: \"kubernetes.io/projected/ad6c338e-0e70-428e-9ba3-2c84deef393b-kube-api-access-8tjtk\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.997473 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997251 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4b1c201-05e2-42c4-9557-0d84c7354b16-registry-certificates\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:49:59.997561 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997536 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad6c338e-0e70-428e-9ba3-2c84deef393b-data-volume\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.997819 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.997800 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad6c338e-0e70-428e-9ba3-2c84deef393b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:49:59.999564 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:49:59.999539 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad6c338e-0e70-428e-9ba3-2c84deef393b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:50:00.005441 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.005416 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tjtk\" (UniqueName: \"kubernetes.io/projected/ad6c338e-0e70-428e-9ba3-2c84deef393b-kube-api-access-8tjtk\") pod \"insights-runtime-extractor-2m6n7\" (UID: \"ad6c338e-0e70-428e-9ba3-2c84deef393b\") " pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:50:00.056270 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.056240 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2m6n7" Apr 20 21:50:00.098655 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098602 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c201-05e2-42c4-9557-0d84c7354b16-trusted-ca\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.098838 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098680 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-registry-tls\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.098838 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098745 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4b1c201-05e2-42c4-9557-0d84c7354b16-registry-certificates\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.098838 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098773 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4b1c201-05e2-42c4-9557-0d84c7354b16-image-registry-private-configuration\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.098838 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098818 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8tkr9\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-kube-api-access-8tkr9\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.099081 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098852 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4b1c201-05e2-42c4-9557-0d84c7354b16-ca-trust-extracted\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.099081 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098876 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4b1c201-05e2-42c4-9557-0d84c7354b16-installation-pull-secrets\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.099081 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.098902 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-bound-sa-token\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.099458 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.099401 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f4b1c201-05e2-42c4-9557-0d84c7354b16-ca-trust-extracted\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.099612 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.099590 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f4b1c201-05e2-42c4-9557-0d84c7354b16-registry-certificates\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.100273 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.100252 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f4b1c201-05e2-42c4-9557-0d84c7354b16-trusted-ca\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.101914 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.101859 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-registry-tls\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.102181 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.102158 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f4b1c201-05e2-42c4-9557-0d84c7354b16-image-registry-private-configuration\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.102526 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.102496 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f4b1c201-05e2-42c4-9557-0d84c7354b16-installation-pull-secrets\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.109526 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.109040 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tkr9\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-kube-api-access-8tkr9\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.110097 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.110048 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f4b1c201-05e2-42c4-9557-0d84c7354b16-bound-sa-token\") pod \"image-registry-c4546fd96-hthl8\" (UID: \"f4b1c201-05e2-42c4-9557-0d84c7354b16\") " pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.180115 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.180079 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2m6n7"] Apr 20 21:50:00.183382 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:00.183351 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6c338e_0e70_428e_9ba3_2c84deef393b.slice/crio-582ee1408609d58dce224b10f664f0c1fb69916f0dad731b0ff1a5807236277a WatchSource:0}: Error finding container 582ee1408609d58dce224b10f664f0c1fb69916f0dad731b0ff1a5807236277a: Status 404 returned error can't find the container with id 582ee1408609d58dce224b10f664f0c1fb69916f0dad731b0ff1a5807236277a Apr 20 21:50:00.193227 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.193202 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:00.323014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:00.322909 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c4546fd96-hthl8"] Apr 20 21:50:00.325332 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:00.325308 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b1c201_05e2_42c4_9557_0d84c7354b16.slice/crio-e8f113c5aaad8a5216eed89b30727437aa2e0a7b08b0ee4fe561a197e547becf WatchSource:0}: Error finding container e8f113c5aaad8a5216eed89b30727437aa2e0a7b08b0ee4fe561a197e547becf: Status 404 returned error can't find the container with id e8f113c5aaad8a5216eed89b30727437aa2e0a7b08b0ee4fe561a197e547becf Apr 20 21:50:01.113711 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.113604 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2m6n7" event={"ID":"ad6c338e-0e70-428e-9ba3-2c84deef393b","Type":"ContainerStarted","Data":"c20fd2bb21507f1807f09233b7a1fe230ef32f9404519320a8041861d8791fc9"} Apr 20 21:50:01.113711 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.113653 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2m6n7" event={"ID":"ad6c338e-0e70-428e-9ba3-2c84deef393b","Type":"ContainerStarted","Data":"6d40ffa36169acf56d7cfb6e760ad2ef6c6043e2e63ddbc8ab6a53a849068794"} Apr 20 21:50:01.113711 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.113663 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2m6n7" event={"ID":"ad6c338e-0e70-428e-9ba3-2c84deef393b","Type":"ContainerStarted","Data":"582ee1408609d58dce224b10f664f0c1fb69916f0dad731b0ff1a5807236277a"} Apr 20 21:50:01.114864 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.114840 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" event={"ID":"f4b1c201-05e2-42c4-9557-0d84c7354b16","Type":"ContainerStarted","Data":"0a2bc18c47eb55355e55aa1e50cf088924f6c0bc5d982b88749855f0fd405946"} Apr 20 21:50:01.114971 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.114871 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" event={"ID":"f4b1c201-05e2-42c4-9557-0d84c7354b16","Type":"ContainerStarted","Data":"e8f113c5aaad8a5216eed89b30727437aa2e0a7b08b0ee4fe561a197e547becf"} Apr 20 21:50:01.115029 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.114972 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:01.134663 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:01.134593 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" podStartSLOduration=2.134576858 podStartE2EDuration="2.134576858s" podCreationTimestamp="2026-04-20 21:49:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 21:50:01.133725217 +0000 UTC m=+168.301045952" watchObservedRunningTime="2026-04-20 21:50:01.134576858 +0000 UTC m=+168.301897590" Apr 20 21:50:02.465607 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:02.465520 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:50:03.123023 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:03.122990 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2m6n7" event={"ID":"ad6c338e-0e70-428e-9ba3-2c84deef393b","Type":"ContainerStarted","Data":"fb2b105284b0623cca61c49bc09148fd99477769f124d1e14d9da94a0ab0d056"} Apr 20 21:50:03.139785 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:03.139733 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2m6n7" podStartSLOduration=2.249749628 podStartE2EDuration="4.139717442s" podCreationTimestamp="2026-04-20 21:49:59 +0000 UTC" firstStartedPulling="2026-04-20 21:50:00.243948139 +0000 UTC m=+167.411268855" lastFinishedPulling="2026-04-20 21:50:02.133915956 +0000 UTC m=+169.301236669" observedRunningTime="2026-04-20 21:50:03.13887703 +0000 UTC m=+170.306197765" watchObservedRunningTime="2026-04-20 21:50:03.139717442 +0000 UTC m=+170.307038177" Apr 20 21:50:03.226929 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:03.226885 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:50:03.229310 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:03.229281 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7561c1f-02bf-450a-9019-4a0e67eb82a3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-75587bd455-hqzpn\" (UID: \"b7561c1f-02bf-450a-9019-4a0e67eb82a3\") " pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:50:03.384954 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:03.384865 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" Apr 20 21:50:03.507451 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:03.507256 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn"] Apr 20 21:50:03.509933 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:03.509903 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7561c1f_02bf_450a_9019_4a0e67eb82a3.slice/crio-9f33a0592dc6cf6fedc6eafb879858ef20dbcad2a503b8bc8032e83867e9dd01 WatchSource:0}: Error finding container 9f33a0592dc6cf6fedc6eafb879858ef20dbcad2a503b8bc8032e83867e9dd01: Status 404 returned error can't find the container with id 9f33a0592dc6cf6fedc6eafb879858ef20dbcad2a503b8bc8032e83867e9dd01 Apr 20 21:50:04.128705 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:04.128662 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" event={"ID":"b7561c1f-02bf-450a-9019-4a0e67eb82a3","Type":"ContainerStarted","Data":"9f33a0592dc6cf6fedc6eafb879858ef20dbcad2a503b8bc8032e83867e9dd01"} Apr 20 21:50:05.574228 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.574191 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5"] Apr 20 21:50:05.577832 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.577807 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:05.580361 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.580336 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-scm92\"" Apr 20 21:50:05.580499 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.580360 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Apr 20 21:50:05.586571 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.586534 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5"] Apr 20 21:50:05.749135 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.749085 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ecda6732-5035-4d99-8333-cef8dbf2e70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jgsj5\" (UID: \"ecda6732-5035-4d99-8333-cef8dbf2e70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:05.849696 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.849599 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ecda6732-5035-4d99-8333-cef8dbf2e70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jgsj5\" (UID: \"ecda6732-5035-4d99-8333-cef8dbf2e70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:05.852266 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.852234 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ecda6732-5035-4d99-8333-cef8dbf2e70a-tls-certificates\") pod \"prometheus-operator-admission-webhook-57cf98b594-jgsj5\" (UID: \"ecda6732-5035-4d99-8333-cef8dbf2e70a\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:05.887874 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:05.887831 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:06.007942 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:06.006579 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5"] Apr 20 21:50:06.012008 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:06.011980 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecda6732_5035_4d99_8333_cef8dbf2e70a.slice/crio-bd23dbb2a6f584d05e10c7bc636d61f80ea48e70dfd5ce3466cdcab7bfb1fb68 WatchSource:0}: Error finding container bd23dbb2a6f584d05e10c7bc636d61f80ea48e70dfd5ce3466cdcab7bfb1fb68: Status 404 returned error can't find the container with id bd23dbb2a6f584d05e10c7bc636d61f80ea48e70dfd5ce3466cdcab7bfb1fb68 Apr 20 21:50:06.134748 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:06.134712 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" event={"ID":"ecda6732-5035-4d99-8333-cef8dbf2e70a","Type":"ContainerStarted","Data":"bd23dbb2a6f584d05e10c7bc636d61f80ea48e70dfd5ce3466cdcab7bfb1fb68"} Apr 20 21:50:06.135941 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:06.135915 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" event={"ID":"b7561c1f-02bf-450a-9019-4a0e67eb82a3","Type":"ContainerStarted","Data":"40ba24ba380ea2166c0dc9711f08c3da60d903c5e87c08d83f434bb78ac2fcca"} Apr 20 21:50:06.152283 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:06.152233 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-75587bd455-hqzpn" podStartSLOduration=33.558296244 podStartE2EDuration="35.15221774s" podCreationTimestamp="2026-04-20 21:49:31 +0000 UTC" firstStartedPulling="2026-04-20 21:50:03.511883354 +0000 UTC m=+170.679204071" lastFinishedPulling="2026-04-20 21:50:05.105804853 +0000 UTC m=+172.273125567" observedRunningTime="2026-04-20 21:50:06.15159157 +0000 UTC m=+173.318912306" watchObservedRunningTime="2026-04-20 21:50:06.15221774 +0000 UTC m=+173.319538476" Apr 20 21:50:07.105143 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:07.105106 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p2pgl" Apr 20 21:50:08.142331 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:08.142292 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" event={"ID":"ecda6732-5035-4d99-8333-cef8dbf2e70a","Type":"ContainerStarted","Data":"0c0c28154e34f96013f32424b995f53a43818f09dfede5a58f8fa23cd2901d60"} Apr 20 21:50:08.142730 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:08.142483 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:08.146972 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:08.146943 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" Apr 20 21:50:08.157444 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:08.157398 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-57cf98b594-jgsj5" podStartSLOduration=2.035549195 podStartE2EDuration="3.157383931s" podCreationTimestamp="2026-04-20 21:50:05 +0000 UTC" firstStartedPulling="2026-04-20 21:50:06.013786288 +0000 UTC m=+173.181107002" lastFinishedPulling="2026-04-20 21:50:07.13562102 +0000 UTC m=+174.302941738" observedRunningTime="2026-04-20 21:50:08.156743483 +0000 UTC m=+175.324064220" watchObservedRunningTime="2026-04-20 21:50:08.157383931 +0000 UTC m=+175.324704667" Apr 20 21:50:12.988961 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.988926 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cnvt4"] Apr 20 21:50:12.992498 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.992475 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:12.994858 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.994832 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-69db897b98-95f7f"] Apr 20 21:50:12.995377 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.995358 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 20 21:50:12.995528 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.995358 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 20 21:50:12.995662 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.995386 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7svp\"" Apr 20 21:50:12.995720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.995423 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 20 21:50:12.996730 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.996704 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 20 21:50:12.998327 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:12.998310 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.002338 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.001708 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Apr 20 21:50:13.002338 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.001725 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Apr 20 21:50:13.002338 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.002024 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qdp8k\"" Apr 20 21:50:13.002338 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.002147 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Apr 20 21:50:13.011866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.011841 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-69db897b98-95f7f"] Apr 20 21:50:13.107078 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107038 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43d52500-1728-4c01-a304-9d73d4447455-metrics-client-ca\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.107078 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107086 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-wtmp\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107112 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-metrics-client-ca\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-textfile\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107240 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43d52500-1728-4c01-a304-9d73d4447455-volume-directive-shadow\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.107304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107265 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7tcg\" (UniqueName: \"kubernetes.io/projected/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-kube-api-access-n7tcg\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107304 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107295 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-tls\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.107480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107314 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.107480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107357 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-sys\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107412 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrhbd\" (UniqueName: \"kubernetes.io/projected/43d52500-1728-4c01-a304-9d73d4447455-kube-api-access-hrhbd\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.107480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107430 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-tls\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107449 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107705 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107509 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-root\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.107705 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.107537 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.208251 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208212 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-textfile\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.208442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208268 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43d52500-1728-4c01-a304-9d73d4447455-volume-directive-shadow\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.208442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208294 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n7tcg\" (UniqueName: \"kubernetes.io/projected/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-kube-api-access-n7tcg\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.208442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208327 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-tls\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.208442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208354 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.208442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208387 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.208442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208416 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-sys\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.208813 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208460 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hrhbd\" (UniqueName: \"kubernetes.io/projected/43d52500-1728-4c01-a304-9d73d4447455-kube-api-access-hrhbd\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.208813 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208552 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-sys\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.208813 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208659 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-textfile\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.208813 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208725 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/43d52500-1728-4c01-a304-9d73d4447455-volume-directive-shadow\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.208813 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208781 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-tls\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209063 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208835 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209063 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208882 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-root\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209063 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208956 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-root\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209063 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.208996 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.209063 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209020 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209063 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209047 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43d52500-1728-4c01-a304-9d73d4447455-metrics-client-ca\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.209340 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209076 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-wtmp\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209340 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209102 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-metrics-client-ca\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209340 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209242 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-wtmp\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209767 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209742 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-metrics-client-ca\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.209878 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209789 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.209999 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.209977 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43d52500-1728-4c01-a304-9d73d4447455-metrics-client-ca\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.211199 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.211172 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.211285 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.211205 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-tls\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.211285 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.211261 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/43d52500-1728-4c01-a304-9d73d4447455-kube-state-metrics-tls\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.211420 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.211311 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.216269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.216238 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrhbd\" (UniqueName: \"kubernetes.io/projected/43d52500-1728-4c01-a304-9d73d4447455-kube-api-access-hrhbd\") pod \"kube-state-metrics-69db897b98-95f7f\" (UID: \"43d52500-1728-4c01-a304-9d73d4447455\") " pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.216363 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.216319 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7tcg\" (UniqueName: \"kubernetes.io/projected/4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4-kube-api-access-n7tcg\") pod \"node-exporter-cnvt4\" (UID: \"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4\") " pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.309405 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.309314 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m7svp\"" Apr 20 21:50:13.316193 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.316145 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qdp8k\"" Apr 20 21:50:13.317174 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.317153 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cnvt4" Apr 20 21:50:13.324533 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.324485 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" Apr 20 21:50:13.474364 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:13.474339 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-69db897b98-95f7f"] Apr 20 21:50:13.476623 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:13.476591 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d52500_1728_4c01_a304_9d73d4447455.slice/crio-5b1af83dfe5f98f631c0b3b03950714489356b13b1bf70900afb554e3f63cbb3 WatchSource:0}: Error finding container 5b1af83dfe5f98f631c0b3b03950714489356b13b1bf70900afb554e3f63cbb3: Status 404 returned error can't find the container with id 5b1af83dfe5f98f631c0b3b03950714489356b13b1bf70900afb554e3f63cbb3 Apr 20 21:50:14.158975 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:14.158939 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" event={"ID":"43d52500-1728-4c01-a304-9d73d4447455","Type":"ContainerStarted","Data":"5b1af83dfe5f98f631c0b3b03950714489356b13b1bf70900afb554e3f63cbb3"} Apr 20 21:50:14.160364 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:14.160332 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnvt4" event={"ID":"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4","Type":"ContainerStarted","Data":"31a8c2651f797ed75812789250214aa10f71040f71918984a29de063e0512701"} Apr 20 21:50:15.164876 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:15.164844 2576 generic.go:358] "Generic (PLEG): container finished" podID="4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4" containerID="f50cad3529616c90112928b86e626c10205ab8e50878dfbd139a69dc140c0330" exitCode=0 Apr 20 21:50:15.165318 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:15.164941 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnvt4" event={"ID":"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4","Type":"ContainerDied","Data":"f50cad3529616c90112928b86e626c10205ab8e50878dfbd139a69dc140c0330"} Apr 20 21:50:15.167021 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:15.166997 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" event={"ID":"43d52500-1728-4c01-a304-9d73d4447455","Type":"ContainerStarted","Data":"040ff7afa3a60719763bbafcfacf0161eaef0026e1895ab0956a5d4920336c1c"} Apr 20 21:50:15.167124 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:15.167029 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" event={"ID":"43d52500-1728-4c01-a304-9d73d4447455","Type":"ContainerStarted","Data":"4d29ad8f5edf0d7687dfd95e646eb106fed5b43e0d1aa4ed7b75bee98f7e995c"} Apr 20 21:50:15.167124 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:15.167043 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" event={"ID":"43d52500-1728-4c01-a304-9d73d4447455","Type":"ContainerStarted","Data":"597bce6afb2167894e2903dfa08c0d3e53fd5037be4628902524a7e6dc8bc233"} Apr 20 21:50:15.199765 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:15.199717 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-69db897b98-95f7f" podStartSLOduration=1.908104778 podStartE2EDuration="3.19970384s" podCreationTimestamp="2026-04-20 21:50:12 +0000 UTC" firstStartedPulling="2026-04-20 21:50:13.478413764 +0000 UTC m=+180.645734477" lastFinishedPulling="2026-04-20 21:50:14.770012825 +0000 UTC m=+181.937333539" observedRunningTime="2026-04-20 21:50:15.198270868 +0000 UTC m=+182.365591603" watchObservedRunningTime="2026-04-20 21:50:15.19970384 +0000 UTC m=+182.367024622" Apr 20 21:50:16.172251 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:16.172200 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnvt4" event={"ID":"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4","Type":"ContainerStarted","Data":"fa38ba3ad864db758913945ada9335caa07de2f6fa44b212fb014171e3968176"} Apr 20 21:50:16.172251 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:16.172250 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cnvt4" event={"ID":"4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4","Type":"ContainerStarted","Data":"c5c66fdc8f4517fc336ac1287dd847214d26bc9d3595ea48643d46458f6e17a5"} Apr 20 21:50:16.196177 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:16.196121 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cnvt4" podStartSLOduration=3.427650318 podStartE2EDuration="4.196107611s" podCreationTimestamp="2026-04-20 21:50:12 +0000 UTC" firstStartedPulling="2026-04-20 21:50:13.337466937 +0000 UTC m=+180.504787665" lastFinishedPulling="2026-04-20 21:50:14.105924241 +0000 UTC m=+181.273244958" observedRunningTime="2026-04-20 21:50:16.194112031 +0000 UTC m=+183.361432771" watchObservedRunningTime="2026-04-20 21:50:16.196107611 +0000 UTC m=+183.363428344" Apr 20 21:50:17.387652 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.387588 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7db7b5f7b-g6d4x"] Apr 20 21:50:17.391028 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.391003 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.393783 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.393756 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Apr 20 21:50:17.393921 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.393757 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Apr 20 21:50:17.393921 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.393909 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-87296\"" Apr 20 21:50:17.394013 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.393961 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Apr 20 21:50:17.395151 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.395133 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Apr 20 21:50:17.395249 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.395165 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c387h4si80103\"" Apr 20 21:50:17.400975 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.400948 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7db7b5f7b-g6d4x"] Apr 20 21:50:17.445445 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445405 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmq74\" (UniqueName: \"kubernetes.io/projected/9038b972-593a-40e6-9806-dbfcb91f540d-kube-api-access-qmq74\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.445617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445460 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038b972-593a-40e6-9806-dbfcb91f540d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.445617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445533 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-secret-metrics-server-tls\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.445617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445598 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9038b972-593a-40e6-9806-dbfcb91f540d-metrics-server-audit-profiles\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.445617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445616 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-client-ca-bundle\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.445810 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445662 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-secret-metrics-server-client-certs\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.445810 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.445710 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9038b972-593a-40e6-9806-dbfcb91f540d-audit-log\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546528 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546490 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038b972-593a-40e6-9806-dbfcb91f540d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546717 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546540 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-secret-metrics-server-tls\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546717 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546598 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9038b972-593a-40e6-9806-dbfcb91f540d-metrics-server-audit-profiles\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546717 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546623 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-client-ca-bundle\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546717 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546669 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-secret-metrics-server-client-certs\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546717 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546703 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9038b972-593a-40e6-9806-dbfcb91f540d-audit-log\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.546960 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.546740 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qmq74\" (UniqueName: \"kubernetes.io/projected/9038b972-593a-40e6-9806-dbfcb91f540d-kube-api-access-qmq74\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.547305 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.547251 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9038b972-593a-40e6-9806-dbfcb91f540d-audit-log\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.547435 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.547411 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038b972-593a-40e6-9806-dbfcb91f540d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.547760 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.547736 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9038b972-593a-40e6-9806-dbfcb91f540d-metrics-server-audit-profiles\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.549246 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.549222 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-secret-metrics-server-tls\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.549353 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.549331 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-client-ca-bundle\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.549417 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.549404 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9038b972-593a-40e6-9806-dbfcb91f540d-secret-metrics-server-client-certs\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.561813 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.561787 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmq74\" (UniqueName: \"kubernetes.io/projected/9038b972-593a-40e6-9806-dbfcb91f540d-kube-api-access-qmq74\") pod \"metrics-server-7db7b5f7b-g6d4x\" (UID: \"9038b972-593a-40e6-9806-dbfcb91f540d\") " pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.701264 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.701181 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:17.819609 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.819571 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx"] Apr 20 21:50:17.824271 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.824248 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:17.826777 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.826751 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Apr 20 21:50:17.826909 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.826774 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-fpgcx\"" Apr 20 21:50:17.832007 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.831979 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx"] Apr 20 21:50:17.841235 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.841213 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7db7b5f7b-g6d4x"] Apr 20 21:50:17.844993 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:17.844966 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9038b972_593a_40e6_9806_dbfcb91f540d.slice/crio-b54dedd10719fa7af675fe30388441069c9786cd47896577abed79ac204ed885 WatchSource:0}: Error finding container b54dedd10719fa7af675fe30388441069c9786cd47896577abed79ac204ed885: Status 404 returned error can't find the container with id b54dedd10719fa7af675fe30388441069c9786cd47896577abed79ac204ed885 Apr 20 21:50:17.849773 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.849749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1108cbd8-2b92-4d00-a7e4-ac1e452db1e8-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-wnhzx\" (UID: \"1108cbd8-2b92-4d00-a7e4-ac1e452db1e8\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:17.951133 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:17.951101 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1108cbd8-2b92-4d00-a7e4-ac1e452db1e8-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-wnhzx\" (UID: \"1108cbd8-2b92-4d00-a7e4-ac1e452db1e8\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:17.951344 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:50:17.951246 2576 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Apr 20 21:50:17.951344 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:50:17.951305 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1108cbd8-2b92-4d00-a7e4-ac1e452db1e8-monitoring-plugin-cert podName:1108cbd8-2b92-4d00-a7e4-ac1e452db1e8 nodeName:}" failed. No retries permitted until 2026-04-20 21:50:18.45128963 +0000 UTC m=+185.618610344 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/1108cbd8-2b92-4d00-a7e4-ac1e452db1e8-monitoring-plugin-cert") pod "monitoring-plugin-7dccd58f55-wnhzx" (UID: "1108cbd8-2b92-4d00-a7e4-ac1e452db1e8") : secret "monitoring-plugin-cert" not found Apr 20 21:50:18.178834 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:18.178795 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" event={"ID":"9038b972-593a-40e6-9806-dbfcb91f540d","Type":"ContainerStarted","Data":"b54dedd10719fa7af675fe30388441069c9786cd47896577abed79ac204ed885"} Apr 20 21:50:18.456784 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:18.456745 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1108cbd8-2b92-4d00-a7e4-ac1e452db1e8-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-wnhzx\" (UID: \"1108cbd8-2b92-4d00-a7e4-ac1e452db1e8\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:18.459579 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:18.459539 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1108cbd8-2b92-4d00-a7e4-ac1e452db1e8-monitoring-plugin-cert\") pod \"monitoring-plugin-7dccd58f55-wnhzx\" (UID: \"1108cbd8-2b92-4d00-a7e4-ac1e452db1e8\") " pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:18.736027 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:18.735939 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:18.872161 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:18.872125 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx"] Apr 20 21:50:19.175127 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:19.175095 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1108cbd8_2b92_4d00_a7e4_ac1e452db1e8.slice/crio-44219a604d629e4b98147deb1dc5cfd6c167b1f8d3b975a8b9ba2b9c8e60dbdb WatchSource:0}: Error finding container 44219a604d629e4b98147deb1dc5cfd6c167b1f8d3b975a8b9ba2b9c8e60dbdb: Status 404 returned error can't find the container with id 44219a604d629e4b98147deb1dc5cfd6c167b1f8d3b975a8b9ba2b9c8e60dbdb Apr 20 21:50:19.182496 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.182465 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" event={"ID":"1108cbd8-2b92-4d00-a7e4-ac1e452db1e8","Type":"ContainerStarted","Data":"44219a604d629e4b98147deb1dc5cfd6c167b1f8d3b975a8b9ba2b9c8e60dbdb"} Apr 20 21:50:19.342706 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.342675 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:50:19.346554 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.346531 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.351443 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.351409 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 20 21:50:19.351614 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.351466 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 20 21:50:19.351614 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.351479 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 20 21:50:19.351614 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.351579 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5p6o9unk6thhc\"" Apr 20 21:50:19.354026 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.354006 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 20 21:50:19.354143 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.354111 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 20 21:50:19.354221 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.354203 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 20 21:50:19.354392 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.354372 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 20 21:50:19.355463 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.355426 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6n4q\"" Apr 20 21:50:19.355463 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.355433 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 20 21:50:19.355827 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.355794 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 20 21:50:19.355912 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.355806 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 20 21:50:19.357400 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.357382 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 20 21:50:19.360035 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.360017 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 20 21:50:19.365671 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.365646 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:50:19.370363 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370341 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370479 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370372 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-web-config\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370479 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370395 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370479 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370418 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370494 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370526 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config-out\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370552 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbjpp\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-kube-api-access-zbjpp\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370722 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370656 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370776 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370724 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370776 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370760 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370871 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370794 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370871 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370822 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370871 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370852 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370973 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370882 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370973 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370913 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370973 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370933 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.370973 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.371091 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.370988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.471885 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.471793 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.471885 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.471851 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.471892 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config-out\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.471919 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbjpp\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-kube-api-access-zbjpp\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.471959 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.471998 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472023 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472053 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472079 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472100 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472122 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472171 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472201 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472226 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472257 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472305 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.472356 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472338 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-web-config\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.473083 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472375 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.473083 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.472912 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.473278 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.473252 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.473442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.473372 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.476312 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.475618 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.476312 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.475669 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.476312 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.476125 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.476528 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.476427 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.476528 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.476521 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config-out\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.477566 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.476841 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.477566 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.477382 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.477566 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.477498 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.478342 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.478318 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.479255 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.479226 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.479357 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.479332 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.479563 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.479539 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.479787 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.479768 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.480797 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.480779 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-web-config\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.484565 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.484539 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbjpp\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-kube-api-access-zbjpp\") pod \"prometheus-k8s-0\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.656341 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.656294 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:19.791989 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:19.791952 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:50:19.796725 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:50:19.796675 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f5f6b0_78a4_41aa_ad72_d8dd0799858e.slice/crio-c59794116bb3c90e559b988e838fb08d1c7107ebd4b0b2825dd8612a124aa58d WatchSource:0}: Error finding container c59794116bb3c90e559b988e838fb08d1c7107ebd4b0b2825dd8612a124aa58d: Status 404 returned error can't find the container with id c59794116bb3c90e559b988e838fb08d1c7107ebd4b0b2825dd8612a124aa58d Apr 20 21:50:20.187843 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:20.187791 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" event={"ID":"9038b972-593a-40e6-9806-dbfcb91f540d","Type":"ContainerStarted","Data":"3182df157404a39c73cada39356af845f64b18fd8e65b4af2aee91e28542c65d"} Apr 20 21:50:20.189347 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:20.189302 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"c59794116bb3c90e559b988e838fb08d1c7107ebd4b0b2825dd8612a124aa58d"} Apr 20 21:50:20.204306 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:20.204240 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" podStartSLOduration=1.83022375 podStartE2EDuration="3.204223141s" podCreationTimestamp="2026-04-20 21:50:17 +0000 UTC" firstStartedPulling="2026-04-20 21:50:17.847074616 +0000 UTC m=+185.014395331" lastFinishedPulling="2026-04-20 21:50:19.221074004 +0000 UTC m=+186.388394722" observedRunningTime="2026-04-20 21:50:20.203613686 +0000 UTC m=+187.370934422" watchObservedRunningTime="2026-04-20 21:50:20.204223141 +0000 UTC m=+187.371543901" Apr 20 21:50:21.193661 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:21.193590 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e" exitCode=0 Apr 20 21:50:21.194081 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:21.193667 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e"} Apr 20 21:50:21.195223 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:21.195193 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" event={"ID":"1108cbd8-2b92-4d00-a7e4-ac1e452db1e8","Type":"ContainerStarted","Data":"756dd040b867278aa5e526f63dd6d073963f9031399eb2e8175cac70c0dcd4ce"} Apr 20 21:50:21.195453 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:21.195403 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:21.200814 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:21.200790 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" Apr 20 21:50:21.235272 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:21.235206 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7dccd58f55-wnhzx" podStartSLOduration=2.612068523 podStartE2EDuration="4.235186495s" podCreationTimestamp="2026-04-20 21:50:17 +0000 UTC" firstStartedPulling="2026-04-20 21:50:19.17753936 +0000 UTC m=+186.344860088" lastFinishedPulling="2026-04-20 21:50:20.800657328 +0000 UTC m=+187.967978060" observedRunningTime="2026-04-20 21:50:21.234918704 +0000 UTC m=+188.402239440" watchObservedRunningTime="2026-04-20 21:50:21.235186495 +0000 UTC m=+188.402507233" Apr 20 21:50:22.123475 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:22.123447 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c4546fd96-hthl8" Apr 20 21:50:24.207234 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:24.207145 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b"} Apr 20 21:50:24.207234 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:24.207184 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa"} Apr 20 21:50:26.215751 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:26.215714 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8"} Apr 20 21:50:26.215751 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:26.215748 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc"} Apr 20 21:50:26.215751 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:26.215758 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9"} Apr 20 21:50:26.216314 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:26.215767 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerStarted","Data":"edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999"} Apr 20 21:50:26.241357 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:26.241306 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.286683544 podStartE2EDuration="7.241291431s" podCreationTimestamp="2026-04-20 21:50:19 +0000 UTC" firstStartedPulling="2026-04-20 21:50:19.799175422 +0000 UTC m=+186.966496136" lastFinishedPulling="2026-04-20 21:50:25.753783305 +0000 UTC m=+192.921104023" observedRunningTime="2026-04-20 21:50:26.239720409 +0000 UTC m=+193.407041142" watchObservedRunningTime="2026-04-20 21:50:26.241291431 +0000 UTC m=+193.408612166" Apr 20 21:50:29.657378 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:29.657328 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:50:37.701591 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:37.701551 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:37.701591 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:37.701596 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:47.281296 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:47.281260 2576 generic.go:358] "Generic (PLEG): container finished" podID="0aaaf0fe-29e0-4335-bb5d-a7f0551f94da" containerID="669d3b8f47a3abf53507b2e3e715e7812fe5420d4766d8ca0c0642e096db42c8" exitCode=0 Apr 20 21:50:47.281778 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:47.281343 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" event={"ID":"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da","Type":"ContainerDied","Data":"669d3b8f47a3abf53507b2e3e715e7812fe5420d4766d8ca0c0642e096db42c8"} Apr 20 21:50:47.281844 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:47.281784 2576 scope.go:117] "RemoveContainer" containerID="669d3b8f47a3abf53507b2e3e715e7812fe5420d4766d8ca0c0642e096db42c8" Apr 20 21:50:47.282863 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:47.282839 2576 generic.go:358] "Generic (PLEG): container finished" podID="c468f16b-183a-4a08-92af-8e224d82525c" containerID="56e7bad4a3c6db55abff97591cf2d0aa706116ee3219ba0af6fb1a169709db1d" exitCode=0 Apr 20 21:50:47.282953 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:47.282896 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" event={"ID":"c468f16b-183a-4a08-92af-8e224d82525c","Type":"ContainerDied","Data":"56e7bad4a3c6db55abff97591cf2d0aa706116ee3219ba0af6fb1a169709db1d"} Apr 20 21:50:47.283245 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:47.283230 2576 scope.go:117] "RemoveContainer" containerID="56e7bad4a3c6db55abff97591cf2d0aa706116ee3219ba0af6fb1a169709db1d" Apr 20 21:50:48.287431 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:48.287397 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6769c5d45-sfsfn" event={"ID":"0aaaf0fe-29e0-4335-bb5d-a7f0551f94da","Type":"ContainerStarted","Data":"2624c252c1f95fff3381268965e4fc39f1a87c5b29fe03f770e78a306ea1e5b3"} Apr 20 21:50:48.289067 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:48.289038 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-d6fc45fc5-qx6f7" event={"ID":"c468f16b-183a-4a08-92af-8e224d82525c","Type":"ContainerStarted","Data":"38ebafe8f210fe12a5da0631fd110f3ad0704e201742764edaf31d840cccbd6f"} Apr 20 21:50:57.707700 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:57.707667 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:50:57.711837 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:50:57.711811 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7db7b5f7b-g6d4x" Apr 20 21:51:11.353861 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:11.353770 2576 generic.go:358] "Generic (PLEG): container finished" podID="05c5b9c5-70a5-43e5-9bb2-864e21b29416" containerID="92a52a4b9650bda1783bffa45634615d60981a8155a137628a978686876e36a7" exitCode=0 Apr 20 21:51:11.353861 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:11.353844 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" event={"ID":"05c5b9c5-70a5-43e5-9bb2-864e21b29416","Type":"ContainerDied","Data":"92a52a4b9650bda1783bffa45634615d60981a8155a137628a978686876e36a7"} Apr 20 21:51:11.354291 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:11.354195 2576 scope.go:117] "RemoveContainer" containerID="92a52a4b9650bda1783bffa45634615d60981a8155a137628a978686876e36a7" Apr 20 21:51:12.359286 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:12.359252 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-585dfdc468-bjk5j" event={"ID":"05c5b9c5-70a5-43e5-9bb2-864e21b29416","Type":"ContainerStarted","Data":"1318e9cce2454af4e01aa78fd95a74797b116da7e59d9a5a728416c6de18953c"} Apr 20 21:51:19.657302 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:19.657261 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:19.677193 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:19.677164 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:20.401347 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:20.401312 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:24.268579 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:24.268482 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:51:24.271090 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:24.271058 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21cfad4e-0887-4313-affc-bf692b73daad-metrics-certs\") pod \"network-metrics-daemon-7b48f\" (UID: \"21cfad4e-0887-4313-affc-bf692b73daad\") " pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:51:24.369489 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:24.369429 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5wzfs\"" Apr 20 21:51:24.376807 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:24.376774 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7b48f" Apr 20 21:51:24.531491 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:24.531386 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7b48f"] Apr 20 21:51:24.534812 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:51:24.534766 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21cfad4e_0887_4313_affc_bf692b73daad.slice/crio-6b370baf602db49e54f12e95d8b48f421fa8f2ce61f3da17856bd909351e2364 WatchSource:0}: Error finding container 6b370baf602db49e54f12e95d8b48f421fa8f2ce61f3da17856bd909351e2364: Status 404 returned error can't find the container with id 6b370baf602db49e54f12e95d8b48f421fa8f2ce61f3da17856bd909351e2364 Apr 20 21:51:25.404478 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:25.404440 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7b48f" event={"ID":"21cfad4e-0887-4313-affc-bf692b73daad","Type":"ContainerStarted","Data":"6b370baf602db49e54f12e95d8b48f421fa8f2ce61f3da17856bd909351e2364"} Apr 20 21:51:26.410269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:26.410217 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7b48f" event={"ID":"21cfad4e-0887-4313-affc-bf692b73daad","Type":"ContainerStarted","Data":"d434bbcdd60dab6d7eb469a6d85c171a9cdff0c5a435121daa0c5783089f71a9"} Apr 20 21:51:26.410269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:26.410263 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7b48f" event={"ID":"21cfad4e-0887-4313-affc-bf692b73daad","Type":"ContainerStarted","Data":"7f0028296068853ea7ac4ad4d98e962b83f0a118120ed12118395abbaa699f98"} Apr 20 21:51:26.428218 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:26.428159 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7b48f" podStartSLOduration=252.311633081 podStartE2EDuration="4m13.428138679s" podCreationTimestamp="2026-04-20 21:47:13 +0000 UTC" firstStartedPulling="2026-04-20 21:51:24.536948783 +0000 UTC m=+251.704269502" lastFinishedPulling="2026-04-20 21:51:25.653454373 +0000 UTC m=+252.820775100" observedRunningTime="2026-04-20 21:51:26.425951161 +0000 UTC m=+253.593271897" watchObservedRunningTime="2026-04-20 21:51:26.428138679 +0000 UTC m=+253.595459418" Apr 20 21:51:37.536793 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.536753 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:51:37.537755 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.537693 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="prometheus" containerID="cri-o://3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa" gracePeriod=600 Apr 20 21:51:37.537902 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.537725 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="thanos-sidecar" containerID="cri-o://edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999" gracePeriod=600 Apr 20 21:51:37.537902 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.537775 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8" gracePeriod=600 Apr 20 21:51:37.537902 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.537727 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-web" containerID="cri-o://f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9" gracePeriod=600 Apr 20 21:51:37.537902 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.537711 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy" containerID="cri-o://b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc" gracePeriod=600 Apr 20 21:51:37.537902 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:37.537742 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="config-reloader" containerID="cri-o://f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b" gracePeriod=600 Apr 20 21:51:38.449736 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449704 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8" exitCode=0 Apr 20 21:51:38.449736 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449729 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc" exitCode=0 Apr 20 21:51:38.449736 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449736 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999" exitCode=0 Apr 20 21:51:38.449736 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449742 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b" exitCode=0 Apr 20 21:51:38.449736 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449747 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa" exitCode=0 Apr 20 21:51:38.450035 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449767 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8"} Apr 20 21:51:38.450035 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449789 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc"} Apr 20 21:51:38.450035 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449798 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999"} Apr 20 21:51:38.450035 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449806 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b"} Apr 20 21:51:38.450035 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.449814 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa"} Apr 20 21:51:38.806058 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.806033 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:38.814856 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814829 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814864 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-db\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814884 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-serving-certs-ca-bundle\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814910 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-thanos-prometheus-http-client-file\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814943 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-trusted-ca-bundle\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814958 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-rulefiles-0\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.814973 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-metrics-client-ca\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815024 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815005 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815047 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-grpc-tls\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815071 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-kubelet-serving-ca-bundle\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815113 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-metrics-client-certs\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815158 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-tls\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815197 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-web-config\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815221 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config-out\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815249 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815280 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-tls-assets\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815330 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbjpp\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-kube-api-access-zbjpp\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815334 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 20 21:51:38.815373 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815356 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-kube-rbac-proxy\") pod \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\" (UID: \"29f5f6b0-78a4-41aa-ad72-d8dd0799858e\") " Apr 20 21:51:38.815913 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815388 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 20 21:51:38.815913 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815673 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.815913 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.815696 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.816216 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.816188 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 20 21:51:38.816966 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.816328 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 20 21:51:38.816966 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.816881 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 20 21:51:38.817152 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.816964 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 20 21:51:38.818337 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.818206 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config" (OuterVolumeSpecName: "config") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.818337 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.818266 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.818337 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.818301 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.818737 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.818695 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.819321 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.819296 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config-out" (OuterVolumeSpecName: "config-out") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 20 21:51:38.819675 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.819594 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 20 21:51:38.819974 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.819949 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.820098 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.820081 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.820592 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.820576 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-kube-api-access-zbjpp" (OuterVolumeSpecName: "kube-api-access-zbjpp") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "kube-api-access-zbjpp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 20 21:51:38.820896 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.820873 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.820998 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.820979 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.831389 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.831318 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-web-config" (OuterVolumeSpecName: "web-config") pod "29f5f6b0-78a4-41aa-ad72-d8dd0799858e" (UID: "29f5f6b0-78a4-41aa-ad72-d8dd0799858e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 20 21:51:38.916621 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916586 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916621 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916619 2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-web-config\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916657 2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config-out\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916667 2576 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-config\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916675 2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-tls-assets\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916684 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbjpp\" (UniqueName: \"kubernetes.io/projected/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-kube-api-access-zbjpp\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916693 2576 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-kube-rbac-proxy\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916704 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916714 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-db\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916722 2576 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916731 2576 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916740 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-metrics-client-ca\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916750 2576 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916759 2576 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-grpc-tls\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916768 2576 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:38.916828 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:38.916777 2576 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/29f5f6b0-78a4-41aa-ad72-d8dd0799858e-secret-metrics-client-certs\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 21:51:39.456061 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.456024 2576 generic.go:358] "Generic (PLEG): container finished" podID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerID="f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9" exitCode=0 Apr 20 21:51:39.456202 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.456105 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9"} Apr 20 21:51:39.456202 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.456149 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"29f5f6b0-78a4-41aa-ad72-d8dd0799858e","Type":"ContainerDied","Data":"c59794116bb3c90e559b988e838fb08d1c7107ebd4b0b2825dd8612a124aa58d"} Apr 20 21:51:39.456202 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.456150 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.456326 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.456210 2576 scope.go:117] "RemoveContainer" containerID="ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8" Apr 20 21:51:39.465353 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.465333 2576 scope.go:117] "RemoveContainer" containerID="b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc" Apr 20 21:51:39.472891 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.472869 2576 scope.go:117] "RemoveContainer" containerID="f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9" Apr 20 21:51:39.480062 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.479974 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:51:39.480947 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.480925 2576 scope.go:117] "RemoveContainer" containerID="edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999" Apr 20 21:51:39.484004 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.483984 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:51:39.488577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.488551 2576 scope.go:117] "RemoveContainer" containerID="f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b" Apr 20 21:51:39.495230 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.495209 2576 scope.go:117] "RemoveContainer" containerID="3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa" Apr 20 21:51:39.502610 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.502590 2576 scope.go:117] "RemoveContainer" containerID="65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e" Apr 20 21:51:39.507332 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507310 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:51:39.507604 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507593 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="thanos-sidecar" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507606 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="thanos-sidecar" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507616 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="config-reloader" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507622 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="config-reloader" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507645 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-web" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507652 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-web" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507663 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="init-config-reloader" Apr 20 21:51:39.507669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507668 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="init-config-reloader" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507676 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507683 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507692 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="prometheus" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507697 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="prometheus" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507703 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-thanos" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507709 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-thanos" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507758 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-thanos" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507767 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507773 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="config-reloader" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507780 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="kube-rbac-proxy-web" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507787 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="prometheus" Apr 20 21:51:39.507866 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.507793 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" containerName="thanos-sidecar" Apr 20 21:51:39.509958 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.509931 2576 scope.go:117] "RemoveContainer" containerID="ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8" Apr 20 21:51:39.510252 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.510232 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8\": container with ID starting with ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8 not found: ID does not exist" containerID="ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8" Apr 20 21:51:39.510312 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510261 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8"} err="failed to get container status \"ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8\": rpc error: code = NotFound desc = could not find container \"ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8\": container with ID starting with ac16436f2f1a4948ce1602bc466b40cbcaea3c911d6c4a4d7b2a3313501b11f8 not found: ID does not exist" Apr 20 21:51:39.510312 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510293 2576 scope.go:117] "RemoveContainer" containerID="b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc" Apr 20 21:51:39.510517 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.510499 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc\": container with ID starting with b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc not found: ID does not exist" containerID="b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc" Apr 20 21:51:39.510560 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510523 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc"} err="failed to get container status \"b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc\": rpc error: code = NotFound desc = could not find container \"b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc\": container with ID starting with b2a9151fdcd59da98e71bc94646f63dbd95bcce935cdd683a14903b741a661dc not found: ID does not exist" Apr 20 21:51:39.510560 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510540 2576 scope.go:117] "RemoveContainer" containerID="f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9" Apr 20 21:51:39.510755 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.510741 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9\": container with ID starting with f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9 not found: ID does not exist" containerID="f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9" Apr 20 21:51:39.510794 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510759 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9"} err="failed to get container status \"f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9\": rpc error: code = NotFound desc = could not find container \"f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9\": container with ID starting with f864c6a9544f7b3bb464d3ad04d8e22c7e695dc88a32da0a883e21b05b87c8c9 not found: ID does not exist" Apr 20 21:51:39.510794 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510771 2576 scope.go:117] "RemoveContainer" containerID="edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999" Apr 20 21:51:39.510962 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.510944 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999\": container with ID starting with edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999 not found: ID does not exist" containerID="edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999" Apr 20 21:51:39.511023 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510970 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999"} err="failed to get container status \"edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999\": rpc error: code = NotFound desc = could not find container \"edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999\": container with ID starting with edeb56b84903d7341662196de505a822b545e826812e8ab87250292b85972999 not found: ID does not exist" Apr 20 21:51:39.511023 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.510989 2576 scope.go:117] "RemoveContainer" containerID="f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b" Apr 20 21:51:39.511203 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.511188 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b\": container with ID starting with f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b not found: ID does not exist" containerID="f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b" Apr 20 21:51:39.511252 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.511207 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b"} err="failed to get container status \"f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b\": rpc error: code = NotFound desc = could not find container \"f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b\": container with ID starting with f6c517f7de4c98098e8aade99a4353146484d7b8774fe81771ebba643dd7cd5b not found: ID does not exist" Apr 20 21:51:39.511252 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.511219 2576 scope.go:117] "RemoveContainer" containerID="3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa" Apr 20 21:51:39.511420 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.511405 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa\": container with ID starting with 3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa not found: ID does not exist" containerID="3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa" Apr 20 21:51:39.511467 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.511423 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa"} err="failed to get container status \"3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa\": rpc error: code = NotFound desc = could not find container \"3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa\": container with ID starting with 3de8f30309e9b294aabab77fbd02c56382879e4ed1118c5f4565002ef775c5aa not found: ID does not exist" Apr 20 21:51:39.511467 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.511434 2576 scope.go:117] "RemoveContainer" containerID="65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e" Apr 20 21:51:39.511611 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:51:39.511596 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e\": container with ID starting with 65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e not found: ID does not exist" containerID="65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e" Apr 20 21:51:39.511670 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.511614 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e"} err="failed to get container status \"65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e\": rpc error: code = NotFound desc = could not find container \"65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e\": container with ID starting with 65825a2163d2beb2716dacd0448c8a8480e852ab199e4bb70d098f8cae2a974e not found: ID does not exist" Apr 20 21:51:39.512485 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.512471 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.514857 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.514838 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Apr 20 21:51:39.515171 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.515157 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-k6n4q\"" Apr 20 21:51:39.515540 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.515523 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5p6o9unk6thhc\"" Apr 20 21:51:39.515669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.515568 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Apr 20 21:51:39.515669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.515576 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Apr 20 21:51:39.515669 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.515655 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Apr 20 21:51:39.516325 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.516313 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Apr 20 21:51:39.516325 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.516319 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Apr 20 21:51:39.516569 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.516553 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Apr 20 21:51:39.516825 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.516805 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Apr 20 21:51:39.516919 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.516871 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Apr 20 21:51:39.516919 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.516889 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Apr 20 21:51:39.518508 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.518487 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Apr 20 21:51:39.524711 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524682 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.524834 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524749 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f382fb-41d4-40ce-8ed0-166571e53ba8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.524834 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524792 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-config\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.524834 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524826 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524880 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524906 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.524938 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525001 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525179 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525034 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525179 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525065 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525179 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525090 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525179 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525179 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525168 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx25b\" (UniqueName: \"kubernetes.io/projected/d3f382fb-41d4-40ce-8ed0-166571e53ba8-kube-api-access-zx25b\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525204 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525244 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525277 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525303 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f382fb-41d4-40ce-8ed0-166571e53ba8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.525381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.525332 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.529587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.529041 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Apr 20 21:51:39.529587 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.529055 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:51:39.626252 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626215 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626252 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626254 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626275 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626297 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626432 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626471 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626507 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626498 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626527 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626580 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zx25b\" (UniqueName: \"kubernetes.io/projected/d3f382fb-41d4-40ce-8ed0-166571e53ba8-kube-api-access-zx25b\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626614 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626681 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626719 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626743 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f382fb-41d4-40ce-8ed0-166571e53ba8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.626790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626767 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627114 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626810 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627114 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626852 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f382fb-41d4-40ce-8ed0-166571e53ba8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627114 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626892 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-config\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627114 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.626922 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627307 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.627112 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627307 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.627191 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.627409 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.627348 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.628042 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.627617 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.630077 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.630052 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.630757 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.630619 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.630757 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.630655 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631341 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631382 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631820 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d3f382fb-41d4-40ce-8ed0-166571e53ba8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631866 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631904 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631917 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3f382fb-41d4-40ce-8ed0-166571e53ba8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631936 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-web-config\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.631945 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631946 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.632381 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.631938 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3f382fb-41d4-40ce-8ed0-166571e53ba8-config-out\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.632611 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.632592 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d3f382fb-41d4-40ce-8ed0-166571e53ba8-config\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.635327 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.635306 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx25b\" (UniqueName: \"kubernetes.io/projected/d3f382fb-41d4-40ce-8ed0-166571e53ba8-kube-api-access-zx25b\") pod \"prometheus-k8s-0\" (UID: \"d3f382fb-41d4-40ce-8ed0-166571e53ba8\") " pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.825480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.825382 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:51:39.959372 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:39.959168 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Apr 20 21:51:39.961389 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:51:39.961356 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f382fb_41d4_40ce_8ed0_166571e53ba8.slice/crio-251fab397818993926c636303270de3383a899eca03ae70a3e22406d41c8e430 WatchSource:0}: Error finding container 251fab397818993926c636303270de3383a899eca03ae70a3e22406d41c8e430: Status 404 returned error can't find the container with id 251fab397818993926c636303270de3383a899eca03ae70a3e22406d41c8e430 Apr 20 21:51:40.461189 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:40.461151 2576 generic.go:358] "Generic (PLEG): container finished" podID="d3f382fb-41d4-40ce-8ed0-166571e53ba8" containerID="144624df45bc81da760fa25c24e4d11983f00023d8b75e495d5454969fb5d5b8" exitCode=0 Apr 20 21:51:40.461362 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:40.461234 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerDied","Data":"144624df45bc81da760fa25c24e4d11983f00023d8b75e495d5454969fb5d5b8"} Apr 20 21:51:40.461362 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:40.461268 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"251fab397818993926c636303270de3383a899eca03ae70a3e22406d41c8e430"} Apr 20 21:51:41.470458 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.470424 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29f5f6b0-78a4-41aa-ad72-d8dd0799858e" path="/var/lib/kubelet/pods/29f5f6b0-78a4-41aa-ad72-d8dd0799858e/volumes" Apr 20 21:51:41.471012 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.470990 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"eaec805d813789ddaced6acea8bcc151554a078129f247a0d8b881559a038f82"} Apr 20 21:51:41.471053 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.471025 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"aacbe0f6f94ab8ae521304f7f13d0994ba5faec86935ec4d3e181193b3c98cce"} Apr 20 21:51:41.471053 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.471041 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"52f47a55e8145ac66b5978e65a7938bb593bd89e013ba38e1d15cb2c19ee127e"} Apr 20 21:51:41.471150 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.471050 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"2fcf788008ea72f97241cf80c9141c2a829cb6efda3680b0adb29257b8ba3664"} Apr 20 21:51:41.471150 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.471061 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"18b1fdcbf74b60cb74245b6d47078c36d44882bffb0df18af613a5b79c32d5c9"} Apr 20 21:51:41.471150 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.471069 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d3f382fb-41d4-40ce-8ed0-166571e53ba8","Type":"ContainerStarted","Data":"4504e147b4dc52bd8485b3c131f27cbcf91cb8fe7d47b3602cd4fa0fd8120896"} Apr 20 21:51:41.502004 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:41.501914 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.501893328 podStartE2EDuration="2.501893328s" podCreationTimestamp="2026-04-20 21:51:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 21:51:41.50125227 +0000 UTC m=+268.668573054" watchObservedRunningTime="2026-04-20 21:51:41.501893328 +0000 UTC m=+268.669214065" Apr 20 21:51:44.826124 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:51:44.826076 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:52:13.337942 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:13.337909 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:52:13.338577 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:13.338556 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:52:13.345320 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:13.345286 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:52:13.345658 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:13.345614 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:52:13.349748 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:13.349705 2576 kubelet.go:1628] "Image garbage collection succeeded" Apr 20 21:52:39.825778 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:39.825740 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:52:39.841268 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:39.841241 2576 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:52:40.668658 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:52:40.668613 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Apr 20 21:53:25.827008 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.826926 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qrt5m"] Apr 20 21:53:25.830242 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.830217 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:25.833211 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.833185 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 20 21:53:25.836575 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.836546 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qrt5m"] Apr 20 21:53:25.939162 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.939120 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-original-pull-secret\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:25.939347 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.939179 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-dbus\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:25.939347 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:25.939224 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-kubelet-config\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.040265 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.040228 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-original-pull-secret\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.040459 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.040277 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-dbus\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.040459 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.040311 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-kubelet-config\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.040459 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.040413 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-kubelet-config\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.040617 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.040488 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-dbus\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.042588 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.042565 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b-original-pull-secret\") pod \"global-pull-secret-syncer-qrt5m\" (UID: \"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b\") " pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.140965 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.140911 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qrt5m" Apr 20 21:53:26.258489 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.258446 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qrt5m"] Apr 20 21:53:26.262595 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:53:26.262560 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d9f324_76d8_4f15_a2d7_c7e1d2b0bd8b.slice/crio-575834ebe1072d60ce9a45302262420264bef179f8b917bb680d8ccae1866fcd WatchSource:0}: Error finding container 575834ebe1072d60ce9a45302262420264bef179f8b917bb680d8ccae1866fcd: Status 404 returned error can't find the container with id 575834ebe1072d60ce9a45302262420264bef179f8b917bb680d8ccae1866fcd Apr 20 21:53:26.264372 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.264355 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 20 21:53:26.788900 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:26.788857 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qrt5m" event={"ID":"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b","Type":"ContainerStarted","Data":"575834ebe1072d60ce9a45302262420264bef179f8b917bb680d8ccae1866fcd"} Apr 20 21:53:30.802264 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:30.802225 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qrt5m" event={"ID":"a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b","Type":"ContainerStarted","Data":"daed4c74bcc001082fe8e38eb0c23b847454d2370a8b6c3fb15a1b64f2cb434c"} Apr 20 21:53:30.815942 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:53:30.815884 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qrt5m" podStartSLOduration=2.080604822 podStartE2EDuration="5.81586892s" podCreationTimestamp="2026-04-20 21:53:25 +0000 UTC" firstStartedPulling="2026-04-20 21:53:26.26448565 +0000 UTC m=+373.431806365" lastFinishedPulling="2026-04-20 21:53:29.999749733 +0000 UTC m=+377.167070463" observedRunningTime="2026-04-20 21:53:30.815028147 +0000 UTC m=+377.982348885" watchObservedRunningTime="2026-04-20 21:53:30.81586892 +0000 UTC m=+377.983189656" Apr 20 21:55:00.083212 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.083175 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd"] Apr 20 21:55:00.085825 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.085800 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.089205 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.089177 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Apr 20 21:55:00.089543 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.089527 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lmzqf\"" Apr 20 21:55:00.089603 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.089528 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Apr 20 21:55:00.089905 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.089886 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Apr 20 21:55:00.090009 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.089995 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Apr 20 21:55:00.102464 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.102438 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd"] Apr 20 21:55:00.166369 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.166331 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ceee586-d04e-4f9b-b259-478063526b0b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.166369 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.166373 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pndjq\" (UniqueName: \"kubernetes.io/projected/8ceee586-d04e-4f9b-b259-478063526b0b-kube-api-access-pndjq\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.166611 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.166441 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ceee586-d04e-4f9b-b259-478063526b0b-webhook-cert\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.267274 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.267235 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ceee586-d04e-4f9b-b259-478063526b0b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.267274 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.267274 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pndjq\" (UniqueName: \"kubernetes.io/projected/8ceee586-d04e-4f9b-b259-478063526b0b-kube-api-access-pndjq\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.267520 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.267319 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ceee586-d04e-4f9b-b259-478063526b0b-webhook-cert\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.269904 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.269875 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ceee586-d04e-4f9b-b259-478063526b0b-webhook-cert\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.270044 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.269876 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ceee586-d04e-4f9b-b259-478063526b0b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.278200 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.278172 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pndjq\" (UniqueName: \"kubernetes.io/projected/8ceee586-d04e-4f9b-b259-478063526b0b-kube-api-access-pndjq\") pod \"opendatahub-operator-controller-manager-f5f47469b-d9gqd\" (UID: \"8ceee586-d04e-4f9b-b259-478063526b0b\") " pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.396600 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.396564 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:00.526121 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:00.526087 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd"] Apr 20 21:55:00.530001 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:55:00.529973 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ceee586_d04e_4f9b_b259_478063526b0b.slice/crio-e01528263fa087330c86c885a8c3dd4e6be36549913b456a5675158d634c37e4 WatchSource:0}: Error finding container e01528263fa087330c86c885a8c3dd4e6be36549913b456a5675158d634c37e4: Status 404 returned error can't find the container with id e01528263fa087330c86c885a8c3dd4e6be36549913b456a5675158d634c37e4 Apr 20 21:55:01.074687 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:01.074615 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" event={"ID":"8ceee586-d04e-4f9b-b259-478063526b0b","Type":"ContainerStarted","Data":"e01528263fa087330c86c885a8c3dd4e6be36549913b456a5675158d634c37e4"} Apr 20 21:55:03.084438 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:03.084402 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" event={"ID":"8ceee586-d04e-4f9b-b259-478063526b0b","Type":"ContainerStarted","Data":"ae3115b497051503c5c6fdd0f9a1e9f3453f7e148f6becd83c2b1657902b2d7a"} Apr 20 21:55:03.084923 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:03.084552 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:03.106272 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:03.106099 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" podStartSLOduration=0.631227701 podStartE2EDuration="3.106080611s" podCreationTimestamp="2026-04-20 21:55:00 +0000 UTC" firstStartedPulling="2026-04-20 21:55:00.531836279 +0000 UTC m=+467.699156993" lastFinishedPulling="2026-04-20 21:55:03.006688976 +0000 UTC m=+470.174009903" observedRunningTime="2026-04-20 21:55:03.10577168 +0000 UTC m=+470.273092417" watchObservedRunningTime="2026-04-20 21:55:03.106080611 +0000 UTC m=+470.273401348" Apr 20 21:55:05.977091 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.977057 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt"] Apr 20 21:55:05.979650 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.979620 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:05.984076 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.984047 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Apr 20 21:55:05.984076 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.984068 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Apr 20 21:55:05.984269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.984101 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jdmz7\"" Apr 20 21:55:05.984269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.984173 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Apr 20 21:55:05.984269 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.984201 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Apr 20 21:55:05.984425 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.984395 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Apr 20 21:55:05.988722 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:05.988699 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt"] Apr 20 21:55:06.020226 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.020192 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/683d52e1-3476-40c6-a0ca-a03e49a826aa-cert\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.020380 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.020237 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/683d52e1-3476-40c6-a0ca-a03e49a826aa-manager-config\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.020380 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.020266 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q96l6\" (UniqueName: \"kubernetes.io/projected/683d52e1-3476-40c6-a0ca-a03e49a826aa-kube-api-access-q96l6\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.020380 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.020307 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/683d52e1-3476-40c6-a0ca-a03e49a826aa-metrics-cert\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.121522 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.121485 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/683d52e1-3476-40c6-a0ca-a03e49a826aa-manager-config\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.121710 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.121529 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q96l6\" (UniqueName: \"kubernetes.io/projected/683d52e1-3476-40c6-a0ca-a03e49a826aa-kube-api-access-q96l6\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.121710 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.121585 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/683d52e1-3476-40c6-a0ca-a03e49a826aa-metrics-cert\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.121710 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.121655 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/683d52e1-3476-40c6-a0ca-a03e49a826aa-cert\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.122133 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.122112 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/683d52e1-3476-40c6-a0ca-a03e49a826aa-manager-config\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.124031 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.124009 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/683d52e1-3476-40c6-a0ca-a03e49a826aa-cert\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.124208 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.124188 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/683d52e1-3476-40c6-a0ca-a03e49a826aa-metrics-cert\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.129914 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.129891 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q96l6\" (UniqueName: \"kubernetes.io/projected/683d52e1-3476-40c6-a0ca-a03e49a826aa-kube-api-access-q96l6\") pod \"lws-controller-manager-796667c6c8-5rllt\" (UID: \"683d52e1-3476-40c6-a0ca-a03e49a826aa\") " pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.289654 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.289530 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:06.412819 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:06.412771 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt"] Apr 20 21:55:06.415124 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:55:06.415096 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683d52e1_3476_40c6_a0ca_a03e49a826aa.slice/crio-c68b36adde9e5b86c2c027325c76558653e42b2636ab9d8e268fd74b2466e009 WatchSource:0}: Error finding container c68b36adde9e5b86c2c027325c76558653e42b2636ab9d8e268fd74b2466e009: Status 404 returned error can't find the container with id c68b36adde9e5b86c2c027325c76558653e42b2636ab9d8e268fd74b2466e009 Apr 20 21:55:07.098480 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:07.098448 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" event={"ID":"683d52e1-3476-40c6-a0ca-a03e49a826aa","Type":"ContainerStarted","Data":"c68b36adde9e5b86c2c027325c76558653e42b2636ab9d8e268fd74b2466e009"} Apr 20 21:55:09.107723 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:09.107690 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" event={"ID":"683d52e1-3476-40c6-a0ca-a03e49a826aa","Type":"ContainerStarted","Data":"18160bf051eff75b7a2c195d41212945db6fbb0a60419b4be8764c698730d400"} Apr 20 21:55:09.108129 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:09.107826 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:09.125099 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:09.125034 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" podStartSLOduration=1.5291455059999999 podStartE2EDuration="4.125013932s" podCreationTimestamp="2026-04-20 21:55:05 +0000 UTC" firstStartedPulling="2026-04-20 21:55:06.416846679 +0000 UTC m=+473.584167393" lastFinishedPulling="2026-04-20 21:55:09.012715105 +0000 UTC m=+476.180035819" observedRunningTime="2026-04-20 21:55:09.124660465 +0000 UTC m=+476.291981198" watchObservedRunningTime="2026-04-20 21:55:09.125013932 +0000 UTC m=+476.292334670" Apr 20 21:55:14.090313 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:14.090282 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f5f47469b-d9gqd" Apr 20 21:55:20.120023 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:20.119991 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-796667c6c8-5rllt" Apr 20 21:55:28.867573 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.867535 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d7f98b469-flt62"] Apr 20 21:55:28.874342 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.874314 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:28.876877 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.876840 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Apr 20 21:55:28.878055 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.878037 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Apr 20 21:55:28.878181 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.878061 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Apr 20 21:55:28.878181 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.878059 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pdkmt\"" Apr 20 21:55:28.878302 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.878059 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Apr 20 21:55:28.884112 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:28.883901 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d7f98b469-flt62"] Apr 20 21:55:29.034783 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.034745 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121d383c-b0fe-446f-a339-65cf1e05a4fc-tls-certs\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.034957 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.034789 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/121d383c-b0fe-446f-a339-65cf1e05a4fc-tmp\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.034957 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.034850 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whgns\" (UniqueName: \"kubernetes.io/projected/121d383c-b0fe-446f-a339-65cf1e05a4fc-kube-api-access-whgns\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.135918 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.135820 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121d383c-b0fe-446f-a339-65cf1e05a4fc-tls-certs\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.135918 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.135862 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/121d383c-b0fe-446f-a339-65cf1e05a4fc-tmp\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.135918 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.135905 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-whgns\" (UniqueName: \"kubernetes.io/projected/121d383c-b0fe-446f-a339-65cf1e05a4fc-kube-api-access-whgns\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.138140 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.138105 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/121d383c-b0fe-446f-a339-65cf1e05a4fc-tmp\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.138402 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.138384 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/121d383c-b0fe-446f-a339-65cf1e05a4fc-tls-certs\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.153148 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.153108 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whgns\" (UniqueName: \"kubernetes.io/projected/121d383c-b0fe-446f-a339-65cf1e05a4fc-kube-api-access-whgns\") pod \"kube-auth-proxy-d7f98b469-flt62\" (UID: \"121d383c-b0fe-446f-a339-65cf1e05a4fc\") " pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.186246 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.186218 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" Apr 20 21:55:29.315888 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:29.315042 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d7f98b469-flt62"] Apr 20 21:55:29.317614 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:55:29.317574 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121d383c_b0fe_446f_a339_65cf1e05a4fc.slice/crio-4c0472bc6ef9066c534c53d64c3273451648309764b71d47f930dd4826c508e3 WatchSource:0}: Error finding container 4c0472bc6ef9066c534c53d64c3273451648309764b71d47f930dd4826c508e3: Status 404 returned error can't find the container with id 4c0472bc6ef9066c534c53d64c3273451648309764b71d47f930dd4826c508e3 Apr 20 21:55:30.185679 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:30.185590 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" event={"ID":"121d383c-b0fe-446f-a339-65cf1e05a4fc","Type":"ContainerStarted","Data":"4c0472bc6ef9066c534c53d64c3273451648309764b71d47f930dd4826c508e3"} Apr 20 21:55:33.197593 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:33.197493 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" event={"ID":"121d383c-b0fe-446f-a339-65cf1e05a4fc","Type":"ContainerStarted","Data":"910b36ed06b718a2681c018241129d91a6218e74c65a2369f0564dd0fce8fc8f"} Apr 20 21:55:33.213698 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:55:33.213648 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d7f98b469-flt62" podStartSLOduration=1.5897326349999998 podStartE2EDuration="5.213613106s" podCreationTimestamp="2026-04-20 21:55:28 +0000 UTC" firstStartedPulling="2026-04-20 21:55:29.319232639 +0000 UTC m=+496.486553367" lastFinishedPulling="2026-04-20 21:55:32.94311312 +0000 UTC m=+500.110433838" observedRunningTime="2026-04-20 21:55:33.212837182 +0000 UTC m=+500.380157925" watchObservedRunningTime="2026-04-20 21:55:33.213613106 +0000 UTC m=+500.380933841" Apr 20 21:56:56.835114 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.835076 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89"] Apr 20 21:56:56.838486 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.838464 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:56.840981 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.840953 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 20 21:56:56.841296 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.841279 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Apr 20 21:56:56.842259 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.842238 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Apr 20 21:56:56.842351 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.842308 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 20 21:56:56.842351 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.842309 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pljh2\"" Apr 20 21:56:56.846880 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.846848 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89"] Apr 20 21:56:56.911705 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.911664 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zhd\" (UniqueName: \"kubernetes.io/projected/f3399df2-a523-4a0e-9a74-782f11787f68-kube-api-access-58zhd\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:56.912040 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.912017 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3399df2-a523-4a0e-9a74-782f11787f68-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:56.912207 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:56.912189 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3399df2-a523-4a0e-9a74-782f11787f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.013015 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.012983 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3399df2-a523-4a0e-9a74-782f11787f68-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.013015 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.013019 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3399df2-a523-4a0e-9a74-782f11787f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.013266 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.013114 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-58zhd\" (UniqueName: \"kubernetes.io/projected/f3399df2-a523-4a0e-9a74-782f11787f68-kube-api-access-58zhd\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.013326 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:56:57.013275 2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Apr 20 21:56:57.013420 ip-10-0-136-102 kubenswrapper[2576]: E0420 21:56:57.013380 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3399df2-a523-4a0e-9a74-782f11787f68-plugin-serving-cert podName:f3399df2-a523-4a0e-9a74-782f11787f68 nodeName:}" failed. No retries permitted until 2026-04-20 21:56:57.51335651 +0000 UTC m=+584.680677228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/f3399df2-a523-4a0e-9a74-782f11787f68-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-8mt89" (UID: "f3399df2-a523-4a0e-9a74-782f11787f68") : secret "plugin-serving-cert" not found Apr 20 21:56:57.013790 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.013768 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f3399df2-a523-4a0e-9a74-782f11787f68-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.032414 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.032381 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zhd\" (UniqueName: \"kubernetes.io/projected/f3399df2-a523-4a0e-9a74-782f11787f68-kube-api-access-58zhd\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.516702 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.516625 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3399df2-a523-4a0e-9a74-782f11787f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.519102 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.519075 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3399df2-a523-4a0e-9a74-782f11787f68-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-8mt89\" (UID: \"f3399df2-a523-4a0e-9a74-782f11787f68\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.756812 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.756765 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" Apr 20 21:56:57.880667 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:57.880497 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89"] Apr 20 21:56:57.883236 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:56:57.883207 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3399df2_a523_4a0e_9a74_782f11787f68.slice/crio-4105ffe40e9ae45e262e801f2cf0792d0d5d900b2b8117a3ac55d9596337cd33 WatchSource:0}: Error finding container 4105ffe40e9ae45e262e801f2cf0792d0d5d900b2b8117a3ac55d9596337cd33: Status 404 returned error can't find the container with id 4105ffe40e9ae45e262e801f2cf0792d0d5d900b2b8117a3ac55d9596337cd33 Apr 20 21:56:58.479720 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:56:58.479684 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" event={"ID":"f3399df2-a523-4a0e-9a74-782f11787f68","Type":"ContainerStarted","Data":"4105ffe40e9ae45e262e801f2cf0792d0d5d900b2b8117a3ac55d9596337cd33"} Apr 20 21:57:21.005014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:21.004988 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:57:21.005014 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:21.004999 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 21:57:21.011593 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:21.011568 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:57:21.011793 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:21.011776 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 21:57:21.566289 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:21.566250 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" event={"ID":"f3399df2-a523-4a0e-9a74-782f11787f68","Type":"ContainerStarted","Data":"7401c433398030ca3496420ee0346b966f6af1a2a450e1a9c25853a07fe3d300"} Apr 20 21:57:21.584255 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:21.584204 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-8mt89" podStartSLOduration=2.08223645 podStartE2EDuration="25.5841879s" podCreationTimestamp="2026-04-20 21:56:56 +0000 UTC" firstStartedPulling="2026-04-20 21:56:57.884487042 +0000 UTC m=+585.051807757" lastFinishedPulling="2026-04-20 21:57:21.386438493 +0000 UTC m=+608.553759207" observedRunningTime="2026-04-20 21:57:21.582282195 +0000 UTC m=+608.749602931" watchObservedRunningTime="2026-04-20 21:57:21.5841879 +0000 UTC m=+608.751508635" Apr 20 21:57:44.820254 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.820209 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:57:44.889898 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.889862 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:57:44.889898 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.889896 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:57:44.890135 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.890026 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:44.892854 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.892831 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Apr 20 21:57:44.993500 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.993461 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8wwx\" (UniqueName: \"kubernetes.io/projected/1585be80-c499-42f1-9144-14ef0c908191-kube-api-access-n8wwx\") pod \"limitador-limitador-78c99df468-dfhzt\" (UID: \"1585be80-c499-42f1-9144-14ef0c908191\") " pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:44.993719 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:44.993586 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1585be80-c499-42f1-9144-14ef0c908191-config-file\") pod \"limitador-limitador-78c99df468-dfhzt\" (UID: \"1585be80-c499-42f1-9144-14ef0c908191\") " pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:45.094106 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.094023 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1585be80-c499-42f1-9144-14ef0c908191-config-file\") pod \"limitador-limitador-78c99df468-dfhzt\" (UID: \"1585be80-c499-42f1-9144-14ef0c908191\") " pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:45.094106 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.094072 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n8wwx\" (UniqueName: \"kubernetes.io/projected/1585be80-c499-42f1-9144-14ef0c908191-kube-api-access-n8wwx\") pod \"limitador-limitador-78c99df468-dfhzt\" (UID: \"1585be80-c499-42f1-9144-14ef0c908191\") " pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:45.100339 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.094970 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1585be80-c499-42f1-9144-14ef0c908191-config-file\") pod \"limitador-limitador-78c99df468-dfhzt\" (UID: \"1585be80-c499-42f1-9144-14ef0c908191\") " pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:45.105722 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.105696 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8wwx\" (UniqueName: \"kubernetes.io/projected/1585be80-c499-42f1-9144-14ef0c908191-kube-api-access-n8wwx\") pod \"limitador-limitador-78c99df468-dfhzt\" (UID: \"1585be80-c499-42f1-9144-14ef0c908191\") " pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:45.200238 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.200197 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:45.332224 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.332119 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:57:45.334978 ip-10-0-136-102 kubenswrapper[2576]: W0420 21:57:45.334951 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1585be80_c499_42f1_9144_14ef0c908191.slice/crio-1dded7ef72d77c887855abf29f04014014766a7798444abdb7643fbd9499de52 WatchSource:0}: Error finding container 1dded7ef72d77c887855abf29f04014014766a7798444abdb7643fbd9499de52: Status 404 returned error can't find the container with id 1dded7ef72d77c887855abf29f04014014766a7798444abdb7643fbd9499de52 Apr 20 21:57:45.648715 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:45.648677 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" event={"ID":"1585be80-c499-42f1-9144-14ef0c908191","Type":"ContainerStarted","Data":"1dded7ef72d77c887855abf29f04014014766a7798444abdb7643fbd9499de52"} Apr 20 21:57:48.662250 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:48.662204 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" event={"ID":"1585be80-c499-42f1-9144-14ef0c908191","Type":"ContainerStarted","Data":"2b5205ed381cddea9188a38405d4af8c3512d99d9ec3e7701cde7991937d528b"} Apr 20 21:57:48.662730 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:48.662262 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:57:48.680442 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:48.680394 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" podStartSLOduration=2.235288541 podStartE2EDuration="4.680380083s" podCreationTimestamp="2026-04-20 21:57:44 +0000 UTC" firstStartedPulling="2026-04-20 21:57:45.336586221 +0000 UTC m=+632.503906935" lastFinishedPulling="2026-04-20 21:57:47.781677761 +0000 UTC m=+634.948998477" observedRunningTime="2026-04-20 21:57:48.679360525 +0000 UTC m=+635.846681262" watchObservedRunningTime="2026-04-20 21:57:48.680380083 +0000 UTC m=+635.847700818" Apr 20 21:57:59.667065 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:57:59.667033 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-dfhzt" Apr 20 21:58:26.822551 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:58:26.822513 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:59:15.168234 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:59:15.168153 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:59:17.879218 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:59:17.879179 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:59:28.577212 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:59:28.577177 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:59:51.971153 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:59:51.971113 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 21:59:59.683744 ip-10-0-136-102 kubenswrapper[2576]: I0420 21:59:59.683705 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:00:15.774083 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:00:15.774047 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:06.473076 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:06.473038 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:10.874713 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:10.874676 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:17.276438 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:17.276398 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:27.671675 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:27.671638 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:36.874837 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:36.874800 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:47.374067 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:47.374032 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:01:56.171187 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:01:56.171085 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:02:06.769517 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:02:06.769480 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:02:21.028729 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:02:21.028701 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:02:21.029954 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:02:21.029931 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:02:21.035342 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:02:21.035321 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:02:21.036180 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:02:21.036159 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:03:08.164781 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:03:08.164747 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:03:23.768821 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:03:23.768738 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:04:02.566084 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:04:02.566045 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:04:19.163229 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:04:19.163190 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:04:33.170855 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:04:33.170824 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:04:49.470896 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:04:49.470858 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:05:17.367127 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:05:17.367036 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:05:21.470956 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:05:21.470925 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:05:43.567889 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:05:43.567859 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:05:52.973146 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:05:52.973073 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:06:08.873329 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:06:08.873287 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:06:18.570451 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:06:18.570422 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:06:34.874678 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:06:34.874576 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:06:42.467276 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:06:42.467240 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:07:15.965370 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:15.965332 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:07:21.053558 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:21.053527 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:07:21.055020 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:21.054996 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:07:21.060239 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:21.060217 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:07:21.061043 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:21.061021 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:07:23.976765 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:23.976725 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:07:32.965401 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:32.965365 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:07:40.863313 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:40.863274 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:07:49.965846 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:07:49.965799 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:08:06.767660 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:08:06.767560 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:08:20.273086 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:08:20.273046 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:06.370893 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:06.370858 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:14.371808 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:14.371771 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:24.066065 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:24.065978 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:32.365327 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:32.365291 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:41.079533 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:41.079491 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:49.778026 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:49.777989 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:09:58.173877 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:09:58.173840 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:07.283652 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:07.283580 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:16.871930 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:16.871890 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:24.773341 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:24.773298 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:33.890025 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:33.889990 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:42.681241 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:42.681199 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:51.780938 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:51.780901 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:10:59.178495 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:10:59.178410 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:11:08.784925 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:11:08.784887 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:11:16.576464 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:11:16.576423 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:11:26.078485 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:11:26.078445 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:11:34.280877 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:11:34.280835 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:12:21.080669 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:12:21.080618 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:12:21.083160 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:12:21.083138 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:12:21.087128 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:12:21.087105 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:12:21.089422 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:12:21.089404 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:13:54.480605 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:13:54.480521 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:13:59.176337 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:13:59.176294 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:14:25.273502 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:14:25.273469 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:14:31.621672 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:14:31.621624 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:14:40.675258 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:14:40.675224 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:14:51.263659 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:14:51.263607 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:14:59.973505 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:14:59.973469 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:00.135560 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.135522 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29612055-jztwr"] Apr 20 22:15:00.138943 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.138920 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:15:00.141541 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.141517 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2zcsj\"" Apr 20 22:15:00.149461 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.149432 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29612055-jztwr"] Apr 20 22:15:00.238264 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.238162 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4l6p\" (UniqueName: \"kubernetes.io/projected/bbabee22-9a41-42ba-8a01-5a30f38ec559-kube-api-access-w4l6p\") pod \"maas-api-key-cleanup-29612055-jztwr\" (UID: \"bbabee22-9a41-42ba-8a01-5a30f38ec559\") " pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:15:00.339053 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.338995 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4l6p\" (UniqueName: \"kubernetes.io/projected/bbabee22-9a41-42ba-8a01-5a30f38ec559-kube-api-access-w4l6p\") pod \"maas-api-key-cleanup-29612055-jztwr\" (UID: \"bbabee22-9a41-42ba-8a01-5a30f38ec559\") " pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:15:00.347026 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.346990 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4l6p\" (UniqueName: \"kubernetes.io/projected/bbabee22-9a41-42ba-8a01-5a30f38ec559-kube-api-access-w4l6p\") pod \"maas-api-key-cleanup-29612055-jztwr\" (UID: \"bbabee22-9a41-42ba-8a01-5a30f38ec559\") " pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:15:00.449078 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.449040 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:15:00.583282 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.583194 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29612055-jztwr"] Apr 20 22:15:00.585867 ip-10-0-136-102 kubenswrapper[2576]: W0420 22:15:00.585832 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbabee22_9a41_42ba_8a01_5a30f38ec559.slice/crio-ba1d5e93afb9b35c4ae737a0c2ca034ec2ae7e171127e7f57e2a08751d354d5c WatchSource:0}: Error finding container ba1d5e93afb9b35c4ae737a0c2ca034ec2ae7e171127e7f57e2a08751d354d5c: Status 404 returned error can't find the container with id ba1d5e93afb9b35c4ae737a0c2ca034ec2ae7e171127e7f57e2a08751d354d5c Apr 20 22:15:00.588067 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:00.588052 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 20 22:15:01.194418 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:01.194379 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerStarted","Data":"ba1d5e93afb9b35c4ae737a0c2ca034ec2ae7e171127e7f57e2a08751d354d5c"} Apr 20 22:15:03.203224 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:03.203189 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerStarted","Data":"563c1ce1f77397210882e036fb54f03b28efd60a3e3c9a31fdc6eaa9c20e8d3f"} Apr 20 22:15:03.217824 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:03.217771 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" podStartSLOduration=1.011708439 podStartE2EDuration="3.217754065s" podCreationTimestamp="2026-04-20 22:15:00 +0000 UTC" firstStartedPulling="2026-04-20 22:15:00.588179963 +0000 UTC m=+1667.755500676" lastFinishedPulling="2026-04-20 22:15:02.794225585 +0000 UTC m=+1669.961546302" observedRunningTime="2026-04-20 22:15:03.217130759 +0000 UTC m=+1670.384451488" watchObservedRunningTime="2026-04-20 22:15:03.217754065 +0000 UTC m=+1670.385074800" Apr 20 22:15:10.660854 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:10.660816 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:19.464866 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:19.464826 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:24.278092 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:24.278058 2576 generic.go:358] "Generic (PLEG): container finished" podID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerID="563c1ce1f77397210882e036fb54f03b28efd60a3e3c9a31fdc6eaa9c20e8d3f" exitCode=6 Apr 20 22:15:24.278472 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:24.278115 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerDied","Data":"563c1ce1f77397210882e036fb54f03b28efd60a3e3c9a31fdc6eaa9c20e8d3f"} Apr 20 22:15:24.278519 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:24.278471 2576 scope.go:117] "RemoveContainer" containerID="563c1ce1f77397210882e036fb54f03b28efd60a3e3c9a31fdc6eaa9c20e8d3f" Apr 20 22:15:25.283700 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:25.283662 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerStarted","Data":"d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3"} Apr 20 22:15:29.963735 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:29.963693 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:39.272500 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:39.272462 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:45.353108 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:45.353069 2576 generic.go:358] "Generic (PLEG): container finished" podID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerID="d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3" exitCode=6 Apr 20 22:15:45.353108 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:45.353111 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerDied","Data":"d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3"} Apr 20 22:15:45.353763 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:45.353147 2576 scope.go:117] "RemoveContainer" containerID="563c1ce1f77397210882e036fb54f03b28efd60a3e3c9a31fdc6eaa9c20e8d3f" Apr 20 22:15:45.353763 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:45.353557 2576 scope.go:117] "RemoveContainer" containerID="d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3" Apr 20 22:15:45.353867 ip-10-0-136-102 kubenswrapper[2576]: E0420 22:15:45.353848 2576 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29612055-jztwr_opendatahub(bbabee22-9a41-42ba-8a01-5a30f38ec559)\"" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" Apr 20 22:15:48.365674 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:48.365621 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:58.062902 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:58.062858 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:15:58.466076 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:58.466043 2576 scope.go:117] "RemoveContainer" containerID="d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3" Apr 20 22:15:59.409725 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:59.409688 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerStarted","Data":"41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634"} Apr 20 22:15:59.492734 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:15:59.492686 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29612055-jztwr"] Apr 20 22:16:00.414988 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:00.414933 2576 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" containerID="cri-o://41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634" gracePeriod=30 Apr 20 22:16:19.265792 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.265763 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:16:19.377745 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.377704 2576 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4l6p\" (UniqueName: \"kubernetes.io/projected/bbabee22-9a41-42ba-8a01-5a30f38ec559-kube-api-access-w4l6p\") pod \"bbabee22-9a41-42ba-8a01-5a30f38ec559\" (UID: \"bbabee22-9a41-42ba-8a01-5a30f38ec559\") " Apr 20 22:16:19.380121 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.380083 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbabee22-9a41-42ba-8a01-5a30f38ec559-kube-api-access-w4l6p" (OuterVolumeSpecName: "kube-api-access-w4l6p") pod "bbabee22-9a41-42ba-8a01-5a30f38ec559" (UID: "bbabee22-9a41-42ba-8a01-5a30f38ec559"). InnerVolumeSpecName "kube-api-access-w4l6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 20 22:16:19.478787 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.478765 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4l6p\" (UniqueName: \"kubernetes.io/projected/bbabee22-9a41-42ba-8a01-5a30f38ec559-kube-api-access-w4l6p\") on node \"ip-10-0-136-102.ec2.internal\" DevicePath \"\"" Apr 20 22:16:19.486671 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.486613 2576 generic.go:358] "Generic (PLEG): container finished" podID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerID="41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634" exitCode=6 Apr 20 22:16:19.486818 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.486674 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerDied","Data":"41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634"} Apr 20 22:16:19.486818 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.486697 2576 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" Apr 20 22:16:19.486818 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.486721 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29612055-jztwr" event={"ID":"bbabee22-9a41-42ba-8a01-5a30f38ec559","Type":"ContainerDied","Data":"ba1d5e93afb9b35c4ae737a0c2ca034ec2ae7e171127e7f57e2a08751d354d5c"} Apr 20 22:16:19.486818 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.486741 2576 scope.go:117] "RemoveContainer" containerID="41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634" Apr 20 22:16:19.495608 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.495584 2576 scope.go:117] "RemoveContainer" containerID="d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3" Apr 20 22:16:19.503043 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.503012 2576 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29612055-jztwr"] Apr 20 22:16:19.503554 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.503534 2576 scope.go:117] "RemoveContainer" containerID="41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634" Apr 20 22:16:19.503876 ip-10-0-136-102 kubenswrapper[2576]: E0420 22:16:19.503857 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634\": container with ID starting with 41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634 not found: ID does not exist" containerID="41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634" Apr 20 22:16:19.504008 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.503884 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634"} err="failed to get container status \"41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634\": rpc error: code = NotFound desc = could not find container \"41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634\": container with ID starting with 41b4df1e5dbcfb23956017d77fa80eb7a98a12619e467835b5d781951e1e2634 not found: ID does not exist" Apr 20 22:16:19.504008 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.503902 2576 scope.go:117] "RemoveContainer" containerID="d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3" Apr 20 22:16:19.504123 ip-10-0-136-102 kubenswrapper[2576]: E0420 22:16:19.504109 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3\": container with ID starting with d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3 not found: ID does not exist" containerID="d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3" Apr 20 22:16:19.504162 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.504126 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3"} err="failed to get container status \"d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3\": rpc error: code = NotFound desc = could not find container \"d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3\": container with ID starting with d43eaeb31ea04a3dc190fa7b1efd60539bf8641405b697bfeaed0dc28161e8a3 not found: ID does not exist" Apr 20 22:16:19.508179 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:19.508150 2576 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-29612055-jztwr"] Apr 20 22:16:21.470584 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:21.470551 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" path="/var/lib/kubelet/pods/bbabee22-9a41-42ba-8a01-5a30f38ec559/volumes" Apr 20 22:16:31.269276 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:16:31.269232 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:17:13.184647 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:13.184587 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:17:21.105048 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:21.105012 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:17:21.108020 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:21.107996 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:17:21.111866 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:21.111838 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:17:21.115411 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:21.115389 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:17:22.468770 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:22.468732 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:17:30.371197 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:30.371161 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:17:38.567966 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:38.567929 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:17:48.596104 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:48.596066 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:17:59.470913 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:17:59.470880 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:18:07.373920 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:18:07.373878 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:18:14.266256 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:18:14.266212 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:18:24.276773 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:18:24.276681 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:18:32.667868 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:18:32.667834 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:18:41.472965 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:18:41.472931 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:18:51.761135 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:18:51.761090 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:19:10.365702 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:19:10.365663 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:19:18.473948 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:19:18.473911 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:19:27.664639 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:19:27.664586 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:19:35.865651 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:19:35.865596 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:19:52.873355 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:19:52.873270 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:00.876970 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:00.876929 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:09.874375 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:09.874335 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:17.206777 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:17.206736 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:27.079074 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:27.079032 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:35.775588 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:35.775545 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:45.086816 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:45.086780 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:20:55.772331 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:20:55.772293 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:21:04.675728 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:21:04.675690 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:21:17.867671 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:21:17.867619 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:21:27.078563 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:21:27.078473 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:21:34.870684 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:21:34.870642 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:21:42.767551 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:21:42.767508 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:21:49.859622 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:21:49.859582 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:22:07.666687 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:07.666649 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:22:15.668920 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:15.668881 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:22:21.130705 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:21.130671 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:22:21.135596 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:21.135563 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:22:21.137556 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:21.137528 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:22:21.141868 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:21.141842 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:22:25.670521 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:25.670484 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:22:33.965966 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:33.965926 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:22:56.370014 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:22:56.369929 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:23:09.167737 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:09.167699 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-dfhzt"] Apr 20 22:23:15.641133 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:15.641095 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f5f47469b-d9gqd_8ceee586-d04e-4f9b-b259-478063526b0b/manager/0.log" Apr 20 22:23:17.597308 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:17.597280 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-8mt89_f3399df2-a523-4a0e-9a74-782f11787f68/kuadrant-console-plugin/0.log" Apr 20 22:23:17.957956 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:17.957923 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-dfhzt_1585be80-c499-42f1-9144-14ef0c908191/limitador/0.log" Apr 20 22:23:18.757703 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:18.757665 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d7f98b469-flt62_121d383c-b0fe-446f-a339-65cf1e05a4fc/kube-auth-proxy/0.log" Apr 20 22:23:23.513973 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.513937 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5l2ks/must-gather-lgwcd"] Apr 20 22:23:23.514493 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514472 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514577 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514496 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514577 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514548 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514577 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514557 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514772 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514652 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514772 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514667 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514772 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514764 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514920 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514775 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.514920 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.514863 2576 memory_manager.go:356] "RemoveStaleState removing state" podUID="bbabee22-9a41-42ba-8a01-5a30f38ec559" containerName="cleanup" Apr 20 22:23:23.517810 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.517788 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.521245 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.521225 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5l2ks\"/\"kube-root-ca.crt\"" Apr 20 22:23:23.522313 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.522282 2576 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5l2ks\"/\"default-dockercfg-wqjtp\"" Apr 20 22:23:23.522313 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.522304 2576 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5l2ks\"/\"openshift-service-ca.crt\"" Apr 20 22:23:23.533593 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.533564 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5l2ks/must-gather-lgwcd"] Apr 20 22:23:23.554753 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.554705 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5e2d5b67-2306-4e3f-abf8-a552741fb461-must-gather-output\") pod \"must-gather-lgwcd\" (UID: \"5e2d5b67-2306-4e3f-abf8-a552741fb461\") " pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.554941 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.554838 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7vbp\" (UniqueName: \"kubernetes.io/projected/5e2d5b67-2306-4e3f-abf8-a552741fb461-kube-api-access-v7vbp\") pod \"must-gather-lgwcd\" (UID: \"5e2d5b67-2306-4e3f-abf8-a552741fb461\") " pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.656035 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.655999 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v7vbp\" (UniqueName: \"kubernetes.io/projected/5e2d5b67-2306-4e3f-abf8-a552741fb461-kube-api-access-v7vbp\") pod \"must-gather-lgwcd\" (UID: \"5e2d5b67-2306-4e3f-abf8-a552741fb461\") " pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.656228 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.656101 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5e2d5b67-2306-4e3f-abf8-a552741fb461-must-gather-output\") pod \"must-gather-lgwcd\" (UID: \"5e2d5b67-2306-4e3f-abf8-a552741fb461\") " pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.656473 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.656448 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5e2d5b67-2306-4e3f-abf8-a552741fb461-must-gather-output\") pod \"must-gather-lgwcd\" (UID: \"5e2d5b67-2306-4e3f-abf8-a552741fb461\") " pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.663723 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.663698 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7vbp\" (UniqueName: \"kubernetes.io/projected/5e2d5b67-2306-4e3f-abf8-a552741fb461-kube-api-access-v7vbp\") pod \"must-gather-lgwcd\" (UID: \"5e2d5b67-2306-4e3f-abf8-a552741fb461\") " pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.826542 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.826448 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5l2ks/must-gather-lgwcd" Apr 20 22:23:23.952267 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.952243 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5l2ks/must-gather-lgwcd"] Apr 20 22:23:23.955100 ip-10-0-136-102 kubenswrapper[2576]: W0420 22:23:23.955073 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e2d5b67_2306_4e3f_abf8_a552741fb461.slice/crio-262a32ce56120abf2bb9bf1ba0ebf560bb7e93627e5a490ba83d3d4f71c2b4f0 WatchSource:0}: Error finding container 262a32ce56120abf2bb9bf1ba0ebf560bb7e93627e5a490ba83d3d4f71c2b4f0: Status 404 returned error can't find the container with id 262a32ce56120abf2bb9bf1ba0ebf560bb7e93627e5a490ba83d3d4f71c2b4f0 Apr 20 22:23:23.956785 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.956770 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 20 22:23:23.967130 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:23.967100 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5l2ks/must-gather-lgwcd" event={"ID":"5e2d5b67-2306-4e3f-abf8-a552741fb461","Type":"ContainerStarted","Data":"262a32ce56120abf2bb9bf1ba0ebf560bb7e93627e5a490ba83d3d4f71c2b4f0"} Apr 20 22:23:24.972719 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:24.972672 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5l2ks/must-gather-lgwcd" event={"ID":"5e2d5b67-2306-4e3f-abf8-a552741fb461","Type":"ContainerStarted","Data":"e180bd8b4aa19e115323c45ee943ddbbf28db6d847791f56516a9355a5cdbf70"} Apr 20 22:23:24.972719 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:24.972727 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5l2ks/must-gather-lgwcd" event={"ID":"5e2d5b67-2306-4e3f-abf8-a552741fb461","Type":"ContainerStarted","Data":"34be9b5d415b340343e81bead59324348de07dcd2eb33f02f0d2dfb3b026c9f9"} Apr 20 22:23:24.989840 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:24.989770 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-5l2ks/must-gather-lgwcd" podStartSLOduration=1.283839852 podStartE2EDuration="1.989749143s" podCreationTimestamp="2026-04-20 22:23:23 +0000 UTC" firstStartedPulling="2026-04-20 22:23:23.956894911 +0000 UTC m=+2171.124215624" lastFinishedPulling="2026-04-20 22:23:24.662804201 +0000 UTC m=+2171.830124915" observedRunningTime="2026-04-20 22:23:24.987134635 +0000 UTC m=+2172.154455372" watchObservedRunningTime="2026-04-20 22:23:24.989749143 +0000 UTC m=+2172.157069881" Apr 20 22:23:26.243956 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:26.243923 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qrt5m_a9d9f324-76d8-4f15-a2d7-c7e1d2b0bd8b/global-pull-secret-syncer/0.log" Apr 20 22:23:26.385378 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:26.385339 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-thbph_3ba13249-2427-41e8-98f4-ad5adbb97a2d/konnectivity-agent/0.log" Apr 20 22:23:26.408303 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:26.408268 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-102.ec2.internal_f3588c42b05d40fa47c543e3dbe98316/haproxy/0.log" Apr 20 22:23:30.969303 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:30.969200 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-8mt89_f3399df2-a523-4a0e-9a74-782f11787f68/kuadrant-console-plugin/0.log" Apr 20 22:23:31.112965 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:31.112867 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-dfhzt_1585be80-c499-42f1-9144-14ef0c908191/limitador/0.log" Apr 20 22:23:32.786988 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:32.786952 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-75587bd455-hqzpn_b7561c1f-02bf-450a-9019-4a0e67eb82a3/cluster-monitoring-operator/0.log" Apr 20 22:23:32.814353 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:32.814315 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-69db897b98-95f7f_43d52500-1728-4c01-a304-9d73d4447455/kube-state-metrics/0.log" Apr 20 22:23:32.833381 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:32.833342 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-69db897b98-95f7f_43d52500-1728-4c01-a304-9d73d4447455/kube-rbac-proxy-main/0.log" Apr 20 22:23:32.854925 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:32.854887 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-69db897b98-95f7f_43d52500-1728-4c01-a304-9d73d4447455/kube-rbac-proxy-self/0.log" Apr 20 22:23:32.883227 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:32.883146 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7db7b5f7b-g6d4x_9038b972-593a-40e6-9806-dbfcb91f540d/metrics-server/0.log" Apr 20 22:23:32.910355 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:32.910323 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7dccd58f55-wnhzx_1108cbd8-2b92-4d00-a7e4-ac1e452db1e8/monitoring-plugin/0.log" Apr 20 22:23:33.085858 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.085818 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnvt4_4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4/node-exporter/0.log" Apr 20 22:23:33.114108 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.114069 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnvt4_4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4/kube-rbac-proxy/0.log" Apr 20 22:23:33.128867 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.128834 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cnvt4_4ca6b6c4-9cc6-4d02-b588-68b29c1af5a4/init-textfile/0.log" Apr 20 22:23:33.228289 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.228204 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/prometheus/0.log" Apr 20 22:23:33.248044 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.248016 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/config-reloader/0.log" Apr 20 22:23:33.271725 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.271697 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/thanos-sidecar/0.log" Apr 20 22:23:33.291378 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.291352 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/kube-rbac-proxy-web/0.log" Apr 20 22:23:33.316414 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.316386 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/kube-rbac-proxy/0.log" Apr 20 22:23:33.340915 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.340881 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/kube-rbac-proxy-thanos/0.log" Apr 20 22:23:33.361784 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.361752 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d3f382fb-41d4-40ce-8ed0-166571e53ba8/init-config-reloader/0.log" Apr 20 22:23:33.430215 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:33.430181 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-57cf98b594-jgsj5_ecda6732-5035-4d99-8333-cef8dbf2e70a/prometheus-operator-admission-webhook/0.log" Apr 20 22:23:34.745013 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:34.744984 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-cb95c66f6-vl97c_03b4777a-708e-40d9-892d-37883a45e306/networking-console-plugin/0.log" Apr 20 22:23:35.057271 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.057180 2576 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk"] Apr 20 22:23:35.061978 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.061950 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.068975 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.068935 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk"] Apr 20 22:23:35.183539 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.183487 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-sys\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.183763 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.183557 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-proc\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.183763 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.183606 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-podres\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.183763 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.183650 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-lib-modules\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.183763 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.183719 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trl5r\" (UniqueName: \"kubernetes.io/projected/370801d9-e6c1-4e60-a882-d655406ccb57-kube-api-access-trl5r\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284317 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284280 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-podres\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284520 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284336 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-lib-modules\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284520 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284399 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-trl5r\" (UniqueName: \"kubernetes.io/projected/370801d9-e6c1-4e60-a882-d655406ccb57-kube-api-access-trl5r\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284520 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284450 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-podres\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284520 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284463 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-sys\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284520 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284513 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-lib-modules\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284769 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284523 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-sys\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284769 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284542 2576 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-proc\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.284769 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.284661 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/370801d9-e6c1-4e60-a882-d655406ccb57-proc\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.292453 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.292413 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trl5r\" (UniqueName: \"kubernetes.io/projected/370801d9-e6c1-4e60-a882-d655406ccb57-kube-api-access-trl5r\") pod \"perf-node-gather-daemonset-r98mk\" (UID: \"370801d9-e6c1-4e60-a882-d655406ccb57\") " pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.297162 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.297107 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/1.log" Apr 20 22:23:35.309319 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.309197 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-9d4b6777b-wwr45_86a52490-01a5-444d-bf45-653d11ab3fc6/console-operator/2.log" Apr 20 22:23:35.376651 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.376600 2576 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:35.537967 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:35.537824 2576 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk"] Apr 20 22:23:35.540947 ip-10-0-136-102 kubenswrapper[2576]: W0420 22:23:35.540905 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod370801d9_e6c1_4e60_a882_d655406ccb57.slice/crio-ef4125539ca2047c7c4cf201f8a03ec3f86b53e5340e259583df8da6e1597b07 WatchSource:0}: Error finding container ef4125539ca2047c7c4cf201f8a03ec3f86b53e5340e259583df8da6e1597b07: Status 404 returned error can't find the container with id ef4125539ca2047c7c4cf201f8a03ec3f86b53e5340e259583df8da6e1597b07 Apr 20 22:23:36.051350 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:36.051312 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" event={"ID":"370801d9-e6c1-4e60-a882-d655406ccb57","Type":"ContainerStarted","Data":"1702ead093ad0fea28807269c7d3ae9e09587816501e3ca5046bec0ae9508930"} Apr 20 22:23:36.051350 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:36.051357 2576 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" event={"ID":"370801d9-e6c1-4e60-a882-d655406ccb57","Type":"ContainerStarted","Data":"ef4125539ca2047c7c4cf201f8a03ec3f86b53e5340e259583df8da6e1597b07"} Apr 20 22:23:36.051826 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:36.051409 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:36.072877 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:36.072800 2576 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" podStartSLOduration=1.072776916 podStartE2EDuration="1.072776916s" podCreationTimestamp="2026-04-20 22:23:35 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-20 22:23:36.071128859 +0000 UTC m=+2183.238449605" watchObservedRunningTime="2026-04-20 22:23:36.072776916 +0000 UTC m=+2183.240097653" Apr 20 22:23:36.261564 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:36.261526 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7c6cbb6c87-lgv4r_34d1dcb2-71b0-4b76-b5b9-9195fb6f4494/volume-data-source-validator/0.log" Apr 20 22:23:37.039992 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:37.039962 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p2pgl_2d733792-4f92-4b9a-8436-4427d99740cd/dns/0.log" Apr 20 22:23:37.057976 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:37.057946 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p2pgl_2d733792-4f92-4b9a-8436-4427d99740cd/kube-rbac-proxy/0.log" Apr 20 22:23:37.145549 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:37.145524 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7t457_e3c32b6a-2f6a-4754-b26a-70179ed0c9eb/dns-node-resolver/0.log" Apr 20 22:23:37.609111 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:37.609082 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-c4546fd96-hthl8_f4b1c201-05e2-42c4-9557-0d84c7354b16/registry/0.log" Apr 20 22:23:37.649581 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:37.649536 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fhmlz_b7c4a314-11cd-4566-9a22-993779800d6c/node-ca/0.log" Apr 20 22:23:38.551068 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:38.551021 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d7f98b469-flt62_121d383c-b0fe-446f-a339-65cf1e05a4fc/kube-auth-proxy/0.log" Apr 20 22:23:39.125567 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:39.125534 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6gm99_cce82b4c-1d52-4715-a259-01734afb2a79/serve-healthcheck-canary/0.log" Apr 20 22:23:39.639280 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:39.639247 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-bjk5j_05c5b9c5-70a5-43e5-9bb2-864e21b29416/insights-operator/1.log" Apr 20 22:23:39.639762 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:39.639520 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-585dfdc468-bjk5j_05c5b9c5-70a5-43e5-9bb2-864e21b29416/insights-operator/0.log" Apr 20 22:23:39.674096 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:39.674062 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2m6n7_ad6c338e-0e70-428e-9ba3-2c84deef393b/kube-rbac-proxy/0.log" Apr 20 22:23:39.693111 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:39.693080 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2m6n7_ad6c338e-0e70-428e-9ba3-2c84deef393b/exporter/0.log" Apr 20 22:23:39.713996 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:39.713967 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2m6n7_ad6c338e-0e70-428e-9ba3-2c84deef393b/extractor/0.log" Apr 20 22:23:41.847335 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:41.847298 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f5f47469b-d9gqd_8ceee586-d04e-4f9b-b259-478063526b0b/manager/0.log" Apr 20 22:23:42.065710 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:42.065683 2576 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5l2ks/perf-node-gather-daemonset-r98mk" Apr 20 22:23:43.030572 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:43.030536 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-796667c6c8-5rllt_683d52e1-3476-40c6-a0ca-a03e49a826aa/manager/0.log" Apr 20 22:23:47.775994 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:47.775961 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-sfsfn_0aaaf0fe-29e0-4335-bb5d-a7f0551f94da/kube-storage-version-migrator-operator/1.log" Apr 20 22:23:47.777856 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:47.777831 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6769c5d45-sfsfn_0aaaf0fe-29e0-4335-bb5d-a7f0551f94da/kube-storage-version-migrator-operator/0.log" Apr 20 22:23:48.833209 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.833155 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4c5hz_cd13167d-30d3-485e-a738-4a54ad946027/kube-multus/0.log" Apr 20 22:23:48.857852 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.857824 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/kube-multus-additional-cni-plugins/0.log" Apr 20 22:23:48.878020 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.877988 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/egress-router-binary-copy/0.log" Apr 20 22:23:48.897044 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.897012 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/cni-plugins/0.log" Apr 20 22:23:48.917169 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.917136 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/bond-cni-plugin/0.log" Apr 20 22:23:48.938836 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.938801 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/routeoverride-cni/0.log" Apr 20 22:23:48.957873 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.957838 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/whereabouts-cni-bincopy/0.log" Apr 20 22:23:48.975758 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:48.975726 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm52p_1e86e08c-b3e5-4b48-8155-c3b112031e05/whereabouts-cni/0.log" Apr 20 22:23:49.358884 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:49.358851 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7b48f_21cfad4e-0887-4313-affc-bf692b73daad/network-metrics-daemon/0.log" Apr 20 22:23:49.377280 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:49.377249 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7b48f_21cfad4e-0887-4313-affc-bf692b73daad/kube-rbac-proxy/0.log" Apr 20 22:23:50.829886 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.829854 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-controller/0.log" Apr 20 22:23:50.845321 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.845288 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/0.log" Apr 20 22:23:50.864923 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.864887 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovn-acl-logging/1.log" Apr 20 22:23:50.885798 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.885757 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/kube-rbac-proxy-node/0.log" Apr 20 22:23:50.908124 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.908093 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/kube-rbac-proxy-ovn-metrics/0.log" Apr 20 22:23:50.924561 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.924534 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/northd/0.log" Apr 20 22:23:50.943021 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.942991 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/nbdb/0.log" Apr 20 22:23:50.961032 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:50.961001 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/sbdb/0.log" Apr 20 22:23:51.132899 ip-10-0-136-102 kubenswrapper[2576]: I0420 22:23:51.132867 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vf7zb_ce4147f9-483f-4069-8c46-f85f87f617a4/ovnkube-controller/0.log"