Jun 11 16:06:00.115934 ip-10-0-128-161 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 11 16:06:00.115964 ip-10-0-128-161 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 11 16:06:00.115973 ip-10-0-128-161 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 11 16:06:00.116125 ip-10-0-128-161 systemd[1]: Failed to start Kubernetes Kubelet. Jun 11 16:06:10.246440 ip-10-0-128-161 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 11 16:06:10.246460 ip-10-0-128-161 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3970444fd9114454b8e374019e82e46e -- Jun 11 16:08:55.523242 ip-10-0-128-161 systemd[1]: Starting Kubernetes Kubelet... Jun 11 16:08:55.905840 ip-10-0-128-161 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:08:55.905840 ip-10-0-128-161 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 16:08:55.905840 ip-10-0-128-161 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:08:55.905840 ip-10-0-128-161 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 16:08:55.905840 ip-10-0-128-161 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:08:55.908878 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.908764 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 16:08:55.913189 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913174 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:55.913189 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913189 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913193 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913196 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913199 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913207 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913210 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913216 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913220 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913223 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913226 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913228 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913233 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913236 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913239 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913243 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913245 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913248 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913251 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913253 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:55.913252 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913256 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913259 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913262 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913265 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913268 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913270 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913273 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913275 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913278 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913280 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913282 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913285 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913287 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913290 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913292 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913295 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913297 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913300 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913303 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913305 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:55.913705 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913308 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913310 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913313 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913315 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913318 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913321 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913323 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913326 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913328 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913330 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913333 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913335 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913337 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913340 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913343 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913346 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913348 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913351 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913355 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:55.914245 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913358 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913361 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913364 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913367 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913369 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913372 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913375 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913377 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913380 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913383 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913386 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913388 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913391 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913394 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913398 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913400 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913402 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913406 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913408 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913411 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:55.914718 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913413 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913415 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913418 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913420 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913423 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913425 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913427 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913434 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913437 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913439 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913442 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913445 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913447 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913450 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913452 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913455 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913457 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913460 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913462 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.913465 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:55.915219 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914182 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914189 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914192 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914195 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914198 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914201 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914203 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914206 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914209 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914211 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914214 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914217 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914220 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914223 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914226 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914228 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914233 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914237 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914240 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:55.915696 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914246 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914249 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914252 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914255 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914258 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914262 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914266 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914271 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914274 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914277 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914279 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914282 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914286 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914289 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914291 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914294 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914297 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914299 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914302 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:55.916172 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914305 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914308 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914310 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914313 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914316 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914318 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914321 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914324 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914326 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914328 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914331 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914334 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914336 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914339 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914342 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914344 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914347 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914349 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914352 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914354 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:55.916677 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914357 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914359 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914362 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914364 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914367 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914369 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914372 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914374 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914376 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914379 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914381 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914386 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914388 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914391 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914394 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914396 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914399 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914401 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914403 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914406 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:55.917178 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914408 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914411 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914413 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914416 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914418 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914421 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914424 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914426 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914429 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914431 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914434 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914437 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914439 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914441 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914444 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914447 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914449 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914451 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914454 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914456 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:55.917662 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914458 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914836 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914842 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914845 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914848 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914851 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914854 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914856 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914859 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914861 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914864 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914867 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914869 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914872 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914875 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914877 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914880 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914882 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914885 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914888 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:55.918165 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914890 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914893 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914895 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914899 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914902 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914905 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914908 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914910 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914913 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914915 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914918 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914920 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914923 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914940 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914943 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914946 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914948 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914951 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914956 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:55.918664 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914959 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914962 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914965 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914968 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914971 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914973 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914975 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914978 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914980 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914983 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914986 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914989 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914991 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914994 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914996 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.914999 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915001 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915004 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915006 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915009 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:55.919147 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915012 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915014 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915017 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915019 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915022 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915025 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915029 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915033 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915035 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915038 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915040 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915043 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915046 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915048 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915051 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915053 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915056 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915058 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915060 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915063 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:55.919625 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915065 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915068 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915073 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915076 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915079 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915082 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915084 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915087 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915090 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915093 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915095 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915098 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915101 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915103 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915106 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915109 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915111 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915114 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915117 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:55.920152 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915120 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915122 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915201 2560 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915208 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915224 2560 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915229 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915234 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915237 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915242 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915246 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915249 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915252 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915255 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915258 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915261 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915265 2560 flags.go:64] FLAG: --cgroup-root="" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915268 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915271 2560 flags.go:64] FLAG: --client-ca-file="" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915274 2560 flags.go:64] FLAG: --cloud-provider="external" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915277 2560 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915282 2560 flags.go:64] FLAG: --cluster-domain="" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915284 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915287 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915290 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 16:08:55.920609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915293 2560 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915297 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915300 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915303 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915306 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915309 2560 flags.go:64] FLAG: --contention-profiling="false" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915311 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915315 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915318 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915321 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915326 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915328 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915331 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915334 2560 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915337 2560 flags.go:64] FLAG: --enable-server="true" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915341 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915345 2560 flags.go:64] FLAG: --event-burst="100" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915348 2560 flags.go:64] FLAG: --event-qps="50" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915351 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915354 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915357 2560 flags.go:64] FLAG: --eviction-hard="" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915360 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915363 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915367 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915370 2560 flags.go:64] FLAG: --eviction-soft="" Jun 11 16:08:55.921194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915373 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915376 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915378 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915381 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915384 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915387 2560 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915390 2560 flags.go:64] FLAG: --feature-gates="" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915394 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915397 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915400 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915403 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915406 2560 flags.go:64] FLAG: --healthz-port="10248" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915409 2560 flags.go:64] FLAG: --help="false" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915412 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-128-161.ec2.internal" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915415 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915418 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915421 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915424 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915428 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915431 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915434 2560 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915437 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915440 2560 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 16:08:55.921816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915443 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915446 2560 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915449 2560 flags.go:64] FLAG: --kube-reserved="" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915452 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915455 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915458 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915460 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915464 2560 flags.go:64] FLAG: --lock-file="" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915467 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915470 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915473 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915478 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915481 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915483 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915486 2560 flags.go:64] FLAG: --logging-format="text" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915489 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915492 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915495 2560 flags.go:64] FLAG: --manifest-url="" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915497 2560 flags.go:64] FLAG: --manifest-url-header="" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915501 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915504 2560 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915508 2560 flags.go:64] FLAG: --max-pods="110" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915511 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915514 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915517 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 16:08:55.922438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915521 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915523 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915526 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915529 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915536 2560 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915539 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915542 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915545 2560 flags.go:64] FLAG: --pod-cidr="" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915549 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915554 2560 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915557 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915560 2560 flags.go:64] FLAG: --pods-per-core="0" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915563 2560 flags.go:64] FLAG: --port="10250" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915566 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915569 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05f2b87be25c563b3" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915574 2560 flags.go:64] FLAG: --qos-reserved="" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915577 2560 flags.go:64] FLAG: --read-only-port="10255" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915579 2560 flags.go:64] FLAG: --register-node="true" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915582 2560 flags.go:64] FLAG: --register-with-taints="" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915586 2560 flags.go:64] FLAG: --registry-burst="10" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915588 2560 flags.go:64] FLAG: --registry-qps="5" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915591 2560 flags.go:64] FLAG: --reserved-cpus="" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915594 2560 flags.go:64] FLAG: --reserved-memory="" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915597 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915600 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 16:08:55.923080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915603 2560 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915606 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915608 2560 flags.go:64] FLAG: --runonce="false" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915611 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915614 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915617 2560 flags.go:64] FLAG: --seccomp-default="false" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915620 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915622 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915625 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915628 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915631 2560 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915634 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915637 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915639 2560 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915642 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915645 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915649 2560 flags.go:64] FLAG: --system-cgroups="" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915651 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915657 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915659 2560 flags.go:64] FLAG: --tls-cert-file="" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915662 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915666 2560 flags.go:64] FLAG: --tls-min-version="" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915670 2560 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915673 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915675 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 16:08:55.923829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915678 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915681 2560 flags.go:64] FLAG: --v="2" Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915685 2560 flags.go:64] FLAG: --version="false" Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915688 2560 flags.go:64] FLAG: --vmodule="" Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915692 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.915695 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915854 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915859 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915863 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915867 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915870 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915874 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915881 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915884 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915887 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915890 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915893 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915895 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915898 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915900 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:55.924475 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915903 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915905 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915908 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915911 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915914 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915917 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915919 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915922 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915924 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915942 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915944 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915947 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915950 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915952 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915956 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915959 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915962 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915965 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915967 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:55.924996 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915970 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915973 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915976 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915978 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915981 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915985 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915987 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915990 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915992 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915995 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.915997 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916000 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916003 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916005 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916007 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916010 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916013 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916015 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916018 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916021 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:55.925468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916023 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916026 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916029 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916032 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916034 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916037 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916039 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916041 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916044 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916046 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916049 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916051 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916054 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916056 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916059 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916061 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916064 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916068 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916070 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916073 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:55.925981 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916075 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916078 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916080 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916082 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916085 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916087 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916090 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916092 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916094 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916098 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916102 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916105 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916108 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916110 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916114 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916116 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916119 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916121 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916124 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916126 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:55.926471 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916129 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916131 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916133 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916136 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916138 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.916140 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.916148 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.923094 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.923111 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.924004 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.924013 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924087 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924092 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924096 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924100 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:55.927019 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924103 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924106 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924109 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924111 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924114 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924116 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924119 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924122 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924124 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924127 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924130 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924133 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924135 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924138 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924140 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924143 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924145 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924148 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924150 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924153 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:55.927450 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924155 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924158 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924161 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924163 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924165 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924171 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924174 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924176 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924179 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924181 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924184 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924186 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924189 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924192 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924194 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924196 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924199 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924202 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924204 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924206 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:55.928161 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924209 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924212 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924214 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924217 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924219 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924222 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924224 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924227 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924229 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924231 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924234 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924236 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924238 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924241 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924243 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924246 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924248 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924251 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924256 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:55.928672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924260 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924263 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924266 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924269 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924272 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924274 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924277 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924280 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924282 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924285 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924288 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924290 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924292 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924295 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924297 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924300 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924302 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924305 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924307 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924310 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:55.929175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924312 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924315 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924317 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924320 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924322 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924324 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924327 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924329 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924332 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924334 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924338 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924342 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924346 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924349 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924352 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:55.929707 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924356 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.924362 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924476 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924481 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924484 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924488 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924491 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924494 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924497 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924500 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924503 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924506 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924508 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924511 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924513 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:55.930109 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924516 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924520 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924523 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924526 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924529 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924532 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924535 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924538 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924541 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924544 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924548 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924550 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924554 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924557 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924559 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924562 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924565 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924567 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924570 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:55.930493 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924572 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924574 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924577 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924579 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924582 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924585 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924588 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924591 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924593 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924596 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924598 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924601 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924603 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924606 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924608 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924611 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924613 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924616 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924618 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924621 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:55.930992 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924623 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924626 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924628 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924631 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924634 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924636 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924639 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924641 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924643 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924646 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924649 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924651 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924654 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924656 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924659 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924661 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924664 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924666 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924669 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:55.931505 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924671 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924673 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924676 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924678 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924680 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924683 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924685 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924688 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924690 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924692 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924695 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924698 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924700 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924703 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924705 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924707 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924710 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924712 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924715 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924717 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:55.932094 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924720 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924722 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924725 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924728 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924730 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924732 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924735 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:55.924737 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.924742 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.924867 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.926863 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.927990 2560 server.go:1019] "Starting client certificate rotation" Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.928094 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 16:08:55.932698 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.928130 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 16:08:55.933058 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.930545 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 16:08:55.936629 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.936607 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 16:08:55.950328 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.950304 2560 log.go:25] "Validated CRI v1 runtime API" Jun 11 16:08:55.953181 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.953164 2560 log.go:25] "Validated CRI v1 image API" Jun 11 16:08:55.956875 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.956849 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 16:08:55.958752 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.958730 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 16:08:55.959275 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.959248 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 83b2c1a9-b444-4ab4-a2fa-76396fd1634b:/dev/nvme0n1p3 e2029719-0641-49fc-b2b2-d46ffb93fa00:/dev/nvme0n1p4] Jun 11 16:08:55.959335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.959275 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 16:08:55.964403 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.964288 2560 manager.go:216] Machine: {Timestamp:2026-06-11 16:08:55.962994414 +0000 UTC m=+0.328348579 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099728 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ebe435d928512ed0d505113019f3e SystemUUID:ec2ebe43-5d92-8512-ed0d-505113019f3e BootID:3970444f-d911-4454-b8e3-74019e82e46e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:65:0b:7c:c7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:65:0b:7c:c7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:5b:96:1d:59:a4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 16:08:55.964403 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.964397 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 16:08:55.964546 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.964534 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 16:08:55.967403 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.967378 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 16:08:55.967543 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.967406 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 16:08:55.968161 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.968150 2560 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 16:08:55.968199 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.968163 2560 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 16:08:55.968199 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.968171 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 16:08:55.968723 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.968713 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 16:08:55.969483 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.969474 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 16:08:55.972256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972244 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 11 16:08:55.972363 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972354 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 16:08:55.972403 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972385 2560 kubelet.go:486] "Attempting to sync node with API server" Jun 11 16:08:55.972432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972403 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 16:08:55.972432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972416 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 16:08:55.972432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972431 2560 kubelet.go:398] "Adding apiserver pod source" Jun 11 16:08:55.972562 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972446 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 16:08:55.972808 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972795 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 16:08:55.972856 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.972821 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 16:08:55.975082 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.975064 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 16:08:55.976754 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.976740 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 16:08:55.976813 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.976769 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 16:08:55.977728 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977716 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977732 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977739 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977744 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977753 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977758 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977766 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977771 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 16:08:55.977778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977778 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 16:08:55.978005 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977783 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 16:08:55.978005 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977791 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 16:08:55.978005 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.977797 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 16:08:55.978686 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.978676 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 16:08:55.978686 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.978685 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 16:08:55.982731 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.982715 2560 server.go:1268] "Started kubelet" Jun 11 16:08:55.982872 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.982845 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 16:08:55.982981 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.982910 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 16:08:55.983088 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.982999 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 16:08:55.984375 ip-10-0-128-161 systemd[1]: Started Kubernetes Kubelet. Jun 11 16:08:55.985060 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.984431 2560 server.go:310] "Adding debug handlers to kubelet server" Jun 11 16:08:55.985060 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.984506 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 16:08:55.989032 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.989015 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 16:08:55.989506 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.989489 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 16:08:55.989610 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.989506 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 16:08:55.989610 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.989590 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:55.990229 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990119 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 16:08:55.990229 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990146 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 16:08:55.990229 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990158 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 16:08:55.990387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990273 2560 reconstruct.go:97] "Volume reconstruction finished" Jun 11 16:08:55.990387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990280 2560 reconciler.go:29] "Reconciler: start to sync state" Jun 11 16:08:55.990387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990363 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 16:08:55.990387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990375 2560 factory.go:55] Registering systemd factory Jun 11 16:08:55.990387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990382 2560 factory.go:223] Registration of the systemd container factory successfully Jun 11 16:08:55.990586 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990569 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 16:08:55.990729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990645 2560 factory.go:153] Registering CRI-O factory Jun 11 16:08:55.990729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990658 2560 factory.go:223] Registration of the crio container factory successfully Jun 11 16:08:55.990729 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.990655 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 16:08:55.990729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990678 2560 factory.go:103] Registering Raw factory Jun 11 16:08:55.990729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.990692 2560 manager.go:1196] Started watching for new ooms in manager Jun 11 16:08:55.991015 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.990669 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 16:08:55.991080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.991068 2560 manager.go:349] Starting recovery of all containers Jun 11 16:08:55.991691 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.991661 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z7j9j" Jun 11 16:08:55.991783 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.990765 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-161.ec2.internal.18b812a12741de4e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-161.ec2.internal,UID:ip-10-0-128-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-161.ec2.internal,},FirstTimestamp:2026-06-11 16:08:55.982693966 +0000 UTC m=+0.348048130,LastTimestamp:2026-06-11 16:08:55.982693966 +0000 UTC m=+0.348048130,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-161.ec2.internal,}" Jun 11 16:08:55.997375 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.997348 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 16:08:55.998052 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:55.998023 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 16:08:55.999805 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.999782 2560 manager.go:354] Recovery completed Jun 11 16:08:55.999973 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:55.999951 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z7j9j" Jun 11 16:08:56.007204 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.007188 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.009010 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.008992 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.009070 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.009024 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.009070 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.009035 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.009514 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.009500 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 16:08:56.009514 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.009508 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.009514 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.009514 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 16:08:56.009664 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.009531 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 11 16:08:56.010153 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.010139 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.010203 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.010167 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.010203 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.010179 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.010650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.010629 2560 policy_none.go:49] "None policy: Start" Jun 11 16:08:56.010709 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.010656 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 16:08:56.010709 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.010670 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 16:08:56.011861 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.011801 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-161.ec2.internal.18b812a128d369d0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-161.ec2.internal,UID:ip-10-0-128-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-161.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-161.ec2.internal,},FirstTimestamp:2026-06-11 16:08:56.009009616 +0000 UTC m=+0.374363780,LastTimestamp:2026-06-11 16:08:56.009009616 +0000 UTC m=+0.374363780,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-161.ec2.internal,}" Jun 11 16:08:56.012146 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.012133 2560 policy_none.go:47] "Start" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053257 2560 manager.go:337] "Starting Device Plugin manager" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.053287 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053296 2560 server.go:87] "Starting device plugin registration server" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053485 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053498 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053608 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053705 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.053715 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.054369 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.054421 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.058468 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.059661 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.059695 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.059710 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 16:08:56.061002 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.059771 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 11 16:08:56.061790 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.061775 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:56.154518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.154497 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.155330 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.155308 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.155434 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.155336 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.155434 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.155347 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.155434 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.155377 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.160621 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.160581 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal"] Jun 11 16:08:56.160674 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.160635 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.161389 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.161375 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.161441 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.161406 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.161441 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.161421 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.162679 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.162668 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.162772 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.162759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.162811 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.162787 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.163356 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.163341 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.163426 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.163372 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.163426 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.163385 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.163426 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.163348 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.163568 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.163433 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.163568 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.163450 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.164529 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.164514 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.164579 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.164544 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:56.165208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.165192 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:56.165282 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.165220 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:56.165282 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.165234 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:56.167350 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.167335 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.167411 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.167362 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-161.ec2.internal\": node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.181123 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.181098 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.191794 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.191776 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62b70f08a0210789f3a7a3206903da55-config\") pod \"kube-apiserver-proxy-ip-10-0-128-161.ec2.internal\" (UID: \"62b70f08a0210789f3a7a3206903da55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.191874 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.191802 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b99e1789793b4e1d0327783c4d1e3cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"1b99e1789793b4e1d0327783c4d1e3cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.191874 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.191820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b99e1789793b4e1d0327783c4d1e3cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"1b99e1789793b4e1d0327783c4d1e3cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.193559 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.193543 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-161.ec2.internal\" not found" node="ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.197968 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.197951 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-161.ec2.internal\" not found" node="ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.281247 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.281211 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.292468 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.292443 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b99e1789793b4e1d0327783c4d1e3cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"1b99e1789793b4e1d0327783c4d1e3cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.292604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.292480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b99e1789793b4e1d0327783c4d1e3cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"1b99e1789793b4e1d0327783c4d1e3cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.292604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.292505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62b70f08a0210789f3a7a3206903da55-config\") pod \"kube-apiserver-proxy-ip-10-0-128-161.ec2.internal\" (UID: \"62b70f08a0210789f3a7a3206903da55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.292604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.292540 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b99e1789793b4e1d0327783c4d1e3cb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"1b99e1789793b4e1d0327783c4d1e3cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.292604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.292547 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/62b70f08a0210789f3a7a3206903da55-config\") pod \"kube-apiserver-proxy-ip-10-0-128-161.ec2.internal\" (UID: \"62b70f08a0210789f3a7a3206903da55\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.292604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.292545 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b99e1789793b4e1d0327783c4d1e3cb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal\" (UID: \"1b99e1789793b4e1d0327783c4d1e3cb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.382348 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.382321 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.482914 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.482839 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.497338 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.497316 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.502201 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.502059 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" Jun 11 16:08:56.583424 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.583395 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.683866 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.683843 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.784401 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.784376 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.884970 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.884942 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:56.894570 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.894554 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:56.928214 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.928195 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 16:08:56.928748 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.928305 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 16:08:56.928748 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.928343 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 16:08:56.960513 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:56.960487 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:56.985644 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:56.985619 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-161.ec2.internal\" not found" Jun 11 16:08:57.005421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.005386 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 16:03:55 +0000 UTC" deadline="2028-01-23 23:02:11.359081527 +0000 UTC" Jun 11 16:08:57.005421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.005420 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14190h53m14.353665774s" Jun 11 16:08:57.034941 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:57.034844 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b70f08a0210789f3a7a3206903da55.slice/crio-2439647681b9df71b5a59f2c5ac12c6f5cd86a53bb6454e8d5c8c51cf3b51389 WatchSource:0}: Error finding container 2439647681b9df71b5a59f2c5ac12c6f5cd86a53bb6454e8d5c8c51cf3b51389: Status 404 returned error can't find the container with id 2439647681b9df71b5a59f2c5ac12c6f5cd86a53bb6454e8d5c8c51cf3b51389 Jun 11 16:08:57.035484 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:57.035457 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b99e1789793b4e1d0327783c4d1e3cb.slice/crio-2b4c941cbe6499dd5eae77e5a8abcdadd1e0b90d02433775200e0e521454428f WatchSource:0}: Error finding container 2b4c941cbe6499dd5eae77e5a8abcdadd1e0b90d02433775200e0e521454428f: Status 404 returned error can't find the container with id 2b4c941cbe6499dd5eae77e5a8abcdadd1e0b90d02433775200e0e521454428f Jun 11 16:08:57.039707 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.039693 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:08:57.065243 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.065195 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" event={"ID":"1b99e1789793b4e1d0327783c4d1e3cb","Type":"ContainerStarted","Data":"2b4c941cbe6499dd5eae77e5a8abcdadd1e0b90d02433775200e0e521454428f"} Jun 11 16:08:57.066195 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.066175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" event={"ID":"62b70f08a0210789f3a7a3206903da55","Type":"ContainerStarted","Data":"2439647681b9df71b5a59f2c5ac12c6f5cd86a53bb6454e8d5c8c51cf3b51389"} Jun 11 16:08:57.078331 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.078315 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:57.090485 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.090469 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" Jun 11 16:08:57.106680 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.106657 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 16:08:57.107806 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.107792 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" Jun 11 16:08:57.119078 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.119057 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 16:08:57.974179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.974143 2560 apiserver.go:52] "Watching apiserver" Jun 11 16:08:57.983438 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.983411 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 16:08:57.983827 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.983801 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-8n7m9","kube-system/global-pull-secret-syncer-xbj5v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg","openshift-dns/node-resolver-275rc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal","openshift-multus/multus-7cb45","openshift-ovn-kubernetes/ovnkube-node-cfvzx","kube-system/konnectivity-agent-mvs4q","kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8t8k5","openshift-image-registry/node-ca-pcqw6","openshift-multus/multus-additional-cni-plugins-qc8kc","openshift-multus/network-metrics-daemon-77sz2","openshift-network-diagnostics/network-check-target-gjz8k"] Jun 11 16:08:57.985708 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.985683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:57.987098 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.986923 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:57.988597 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.988237 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:57.989527 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.989512 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:57.989964 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.989951 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 16:08:57.990900 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.990703 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:08:57.990900 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:57.990774 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:08:57.992351 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.992328 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7cb45" Jun 11 16:08:57.993405 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.993389 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:57.994685 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.994626 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:57.994728 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:57.994709 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:08:57.995380 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.995359 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 16:08:57.995471 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.995359 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 16:08:57.995471 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.995405 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 16:08:57.995584 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.995498 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 16:08:57.996029 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996011 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:57.996126 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:57.996083 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:08:57.996415 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996397 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 16:08:57.996415 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996404 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 16:08:57.996640 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996628 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 16:08:57.996759 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996742 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 16:08:57.996759 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996754 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 16:08:57.996949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996900 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 16:08:57.997020 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.996955 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gztfr\"" Jun 11 16:08:57.997125 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997096 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vdnpw\"" Jun 11 16:08:57.997221 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997186 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 16:08:57.997221 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997206 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997614 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997699 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997783 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w6frk\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997794 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g2xvx\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997800 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997953 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.997973 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 16:08:57.998111 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.998092 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 16:08:57.998459 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.998118 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 16:08:57.998459 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.998257 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 16:08:57.998761 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.998741 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 16:08:58.000967 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.999214 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.000967 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.999328 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lrcbg\"" Jun 11 16:08:58.000967 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:57.999495 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5khmn\"" Jun 11 16:08:58.000967 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.000888 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxxq\" (UniqueName: \"kubernetes.io/projected/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-kube-api-access-8nxxq\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.000967 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.000946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.000979 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-ovn\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001009 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-kubelet\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-run-ovn-kubernetes\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001099 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:08:58.001257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001167 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-cni-netd\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001225 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5pcw\" (UniqueName: \"kubernetes.io/projected/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-kube-api-access-m5pcw\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001285 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-slash\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001306 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovnkube-config\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001390 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-cni-bin\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-env-overrides\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovn-node-metrics-cert\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001465 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovnkube-script-lib\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-host\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001514 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-socket-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.001539 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001533 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-device-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001550 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-etc-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001567 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-systemd-units\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-serviceca\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001603 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-etc-selinux\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad8f72cc-af12-44c9-9715-c07563601aee-hosts-file\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad8f72cc-af12-44c9-9715-c07563601aee-tmp-dir\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001659 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pntch\" (UniqueName: \"kubernetes.io/projected/ad8f72cc-af12-44c9-9715-c07563601aee-kube-api-access-pntch\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-run-netns\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-registration-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.002087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.001905 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-sys-fs\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.002518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.002399 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g79gt\" (UniqueName: \"kubernetes.io/projected/28a1ea03-adc1-427e-9aa8-f476cae3424b-kube-api-access-g79gt\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.002518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.002426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-log-socket\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.002518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.002449 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-systemd\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.002518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.002466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-var-lib-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.002518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.002483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-node-log\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.004700 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.004681 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s77mk\"" Jun 11 16:08:58.004903 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.004888 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 16:08:58.009617 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.009600 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qhsp7\"" Jun 11 16:08:58.009704 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.009636 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jgkxs\"" Jun 11 16:08:58.009878 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.009861 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 16:08:58.009956 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.009867 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:08:58.010133 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.010116 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 16:08:58.010984 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.010912 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 16:08:58.011827 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.011808 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 16:08:58.014234 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.014215 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:08:58.014885 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.014863 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 16:08:58.030399 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.030381 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:58.074888 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.074862 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nmr9g" Jun 11 16:08:58.091275 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.091246 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 16:08:58.092741 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.092722 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nmr9g" Jun 11 16:08:58.103220 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-host-slash\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.103334 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103225 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfgjg\" (UniqueName: \"kubernetes.io/projected/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-kube-api-access-kfgjg\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.103334 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.103334 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5pcw\" (UniqueName: \"kubernetes.io/projected/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-kube-api-access-m5pcw\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.103496 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-tuned\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.103550 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-cnibin\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.103603 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103546 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-kubelet\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.103659 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103591 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-conf-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.103731 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-slash\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.103842 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103820 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-slash\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.103912 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103878 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.103912 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.104023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovnkube-config\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.104023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103953 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbknq\" (UniqueName: \"kubernetes.io/projected/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-kube-api-access-jbknq\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.104023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-cni-bin\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.104023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.103988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovn-node-metrics-cert\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.104023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovnkube-script-lib\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.104023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104017 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-host\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.104290 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104033 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-socket-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.104290 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104054 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-device-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.104290 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad8f72cc-af12-44c9-9715-c07563601aee-hosts-file\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.104290 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104131 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad8f72cc-af12-44c9-9715-c07563601aee-hosts-file\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.104290 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104170 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-host\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.104528 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104401 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-socket-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.104528 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104463 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-device-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.104656 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104628 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovnkube-script-lib\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.104706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-iptables-alerter-script\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.104762 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-etc-selinux\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.104762 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad8f72cc-af12-44c9-9715-c07563601aee-tmp-dir\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.104863 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-cni-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.104863 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104796 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-cni-binary-copy\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.104863 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.104863 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-registration-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.104863 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104807 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovnkube-config\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104868 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g79gt\" (UniqueName: \"kubernetes.io/projected/28a1ea03-adc1-427e-9aa8-f476cae3424b-kube-api-access-g79gt\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-k8s-cni-cncf-io\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104982 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0ff4b19f-3067-44b8-bf80-4a7041a91297-kubelet-config\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.104993 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-cni-bin\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-systemd\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-var-lib-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-modprobe-d\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.105115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105063 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 16:08:58.105564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105157 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-etc-selinux\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.105564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105316 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-registration-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.105564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105405 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad8f72cc-af12-44c9-9715-c07563601aee-tmp-dir\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.105564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105459 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-var-lib-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105502 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-systemd\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-run\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.105763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-socket-dir-parent\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.105763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105603 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-node-log\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105626 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-lib-modules\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.105763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwgcn\" (UniqueName: \"kubernetes.io/projected/92164eac-e970-48e0-88bf-5bbcb6554bef-kube-api-access-mwgcn\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.105763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-multus-certs\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.105763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0ff4b19f-3067-44b8-bf80-4a7041a91297-dbus\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105753 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxxq\" (UniqueName: \"kubernetes.io/projected/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-kube-api-access-8nxxq\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-kubelet\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105824 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-run-ovn-kubernetes\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-tmp\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105868 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89d25600-3147-44e1-81e3-46f44c27e2ba-agent-certs\") pod \"konnectivity-agent-mvs4q\" (UID: \"89d25600-3147-44e1-81e3-46f44c27e2ba\") " pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-etc-kubernetes\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-cni-netd\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105920 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-node-log\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.105974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysctl-conf\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.105983 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-system-cni-dir\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-cni-binary-copy\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106031 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-system-cni-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106082 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-cni-bin\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-hostroot\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-kubernetes\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106147 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-daemon-config\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pct8\" (UniqueName: \"kubernetes.io/projected/72260f98-ae30-4e5d-ac3f-989e32672507-kube-api-access-6pct8\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-host\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106214 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-kubelet\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-env-overrides\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106285 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pntch\" (UniqueName: \"kubernetes.io/projected/ad8f72cc-af12-44c9-9715-c07563601aee-kube-api-access-pntch\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106293 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-cni-netd\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.106326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-systemd\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-cni-multus\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106467 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-run-ovn-kubernetes\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106519 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-sys\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106542 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-etc-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-systemd-units\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-serviceca\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106607 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-run-netns\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106630 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-sys-fs\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-var-lib-kubelet\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106730 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/28a1ea03-adc1-427e-9aa8-f476cae3424b-sys-fs\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-cnibin\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-log-socket\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysconfig\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdm68\" (UniqueName: \"kubernetes.io/projected/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-kube-api-access-gdm68\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106860 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89d25600-3147-44e1-81e3-46f44c27e2ba-konnectivity-ca\") pod \"konnectivity-agent-mvs4q\" (UID: \"89d25600-3147-44e1-81e3-46f44c27e2ba\") " pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.107090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-os-release\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106903 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-env-overrides\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106906 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-os-release\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysctl-d\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106970 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-etc-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.106998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-netns\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107005 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-systemd-units\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107057 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-ovn\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107059 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-log-socket\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107396 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-host-run-netns\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107407 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-serviceca\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107425 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-openvswitch\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.107784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.107457 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-run-ovn\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.109116 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.109099 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-ovn-node-metrics-cert\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.118726 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.118705 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5pcw\" (UniqueName: \"kubernetes.io/projected/ce8d71cd-2b84-44af-8bb1-bf8f0daafe26-kube-api-access-m5pcw\") pod \"ovnkube-node-cfvzx\" (UID: \"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26\") " pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.119364 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.119193 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:58.119364 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.119218 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:58.119364 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.119233 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:58.119364 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.119306 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:08:58.619284354 +0000 UTC m=+2.984638507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:58.119555 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.119534 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxxq\" (UniqueName: \"kubernetes.io/projected/03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf-kube-api-access-8nxxq\") pod \"node-ca-pcqw6\" (UID: \"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf\") " pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.120718 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.120694 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pntch\" (UniqueName: \"kubernetes.io/projected/ad8f72cc-af12-44c9-9715-c07563601aee-kube-api-access-pntch\") pod \"node-resolver-275rc\" (UID: \"ad8f72cc-af12-44c9-9715-c07563601aee\") " pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.121059 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.121036 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g79gt\" (UniqueName: \"kubernetes.io/projected/28a1ea03-adc1-427e-9aa8-f476cae3424b-kube-api-access-g79gt\") pod \"aws-ebs-csi-driver-node-59jvg\" (UID: \"28a1ea03-adc1-427e-9aa8-f476cae3424b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.208202 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208174 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-lib-modules\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwgcn\" (UniqueName: \"kubernetes.io/projected/92164eac-e970-48e0-88bf-5bbcb6554bef-kube-api-access-mwgcn\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208239 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-multus-certs\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0ff4b19f-3067-44b8-bf80-4a7041a91297-dbus\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208282 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-tmp\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89d25600-3147-44e1-81e3-46f44c27e2ba-agent-certs\") pod \"konnectivity-agent-mvs4q\" (UID: \"89d25600-3147-44e1-81e3-46f44c27e2ba\") " pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-etc-kubernetes\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208352 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208332 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-lib-modules\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208343 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysctl-conf\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208386 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-system-cni-dir\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-cni-binary-copy\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208426 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-system-cni-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-cni-bin\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208463 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysctl-conf\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-hostroot\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208498 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-system-cni-dir\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-kubernetes\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-daemon-config\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pct8\" (UniqueName: \"kubernetes.io/projected/72260f98-ae30-4e5d-ac3f-989e32672507-kube-api-access-6pct8\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-host\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-systemd\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-cni-multus\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.208682 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208680 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-sys\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-var-lib-kubelet\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-cnibin\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysconfig\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdm68\" (UniqueName: \"kubernetes.io/projected/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-kube-api-access-gdm68\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208809 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89d25600-3147-44e1-81e3-46f44c27e2ba-konnectivity-ca\") pod \"konnectivity-agent-mvs4q\" (UID: \"89d25600-3147-44e1-81e3-46f44c27e2ba\") " pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-os-release\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208842 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-multus-certs\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-os-release\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysctl-d\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208909 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-netns\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208952 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-system-cni-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-host-slash\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208968 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-cni-binary-copy\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.208996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfgjg\" (UniqueName: \"kubernetes.io/projected/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-kube-api-access-kfgjg\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209013 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-sys\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209027 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-cni-bin\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.209511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209078 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-tuned\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209083 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0ff4b19f-3067-44b8-bf80-4a7041a91297-dbus\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-cnibin\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-kubelet\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209146 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-os-release\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-conf-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbknq\" (UniqueName: \"kubernetes.io/projected/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-kube-api-access-jbknq\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209202 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-var-lib-kubelet\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-iptables-alerter-script\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209246 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-cnibin\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209297 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-cni-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209339 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysconfig\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209527 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209596 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92164eac-e970-48e0-88bf-5bbcb6554bef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209618 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-os-release\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209665 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-netns\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209669 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-host-slash\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.210309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209759 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-sysctl-d\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-cni-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209803 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-cni-binary-copy\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209839 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-hostroot\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-k8s-cni-cncf-io\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209949 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-etc-kubernetes\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0ff4b19f-3067-44b8-bf80-4a7041a91297-kubelet-config\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209980 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/89d25600-3147-44e1-81e3-46f44c27e2ba-konnectivity-ca\") pod \"konnectivity-agent-mvs4q\" (UID: \"89d25600-3147-44e1-81e3-46f44c27e2ba\") " pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.209996 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-modprobe-d\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210002 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0ff4b19f-3067-44b8-bf80-4a7041a91297-kubelet-config\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210065 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-host\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210087 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-modprobe-d\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.210093 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210117 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-systemd\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210121 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-run-k8s-cni-cncf-io\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.210157 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:58.710138228 +0000 UTC m=+3.075492378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-run\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-socket-dir-parent\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210280 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92164eac-e970-48e0-88bf-5bbcb6554bef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210288 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-socket-dir-parent\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210329 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-kubernetes\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.210411 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.210466 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:58.710450059 +0000 UTC m=+3.075804213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210513 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-run\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210526 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-cni-multus\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-conf-dir\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210589 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-multus-daemon-config\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210596 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-cnibin\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.210601 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-host-var-lib-kubelet\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.211086 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-cni-binary-copy\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.211155 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-iptables-alerter-script\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.211889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.211869 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-tmp\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.213325 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.213302 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/89d25600-3147-44e1-81e3-46f44c27e2ba-agent-certs\") pod \"konnectivity-agent-mvs4q\" (UID: \"89d25600-3147-44e1-81e3-46f44c27e2ba\") " pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.217349 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.217329 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-etc-tuned\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.224363 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.224309 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdm68\" (UniqueName: \"kubernetes.io/projected/e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1-kube-api-access-gdm68\") pod \"tuned-8t8k5\" (UID: \"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1\") " pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.227841 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.227819 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbknq\" (UniqueName: \"kubernetes.io/projected/2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e-kube-api-access-jbknq\") pod \"multus-7cb45\" (UID: \"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e\") " pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.227971 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.227952 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfgjg\" (UniqueName: \"kubernetes.io/projected/cfcca9f7-3e2f-452e-b19a-4e311fd1e05f-kube-api-access-kfgjg\") pod \"iptables-alerter-8n7m9\" (UID: \"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f\") " pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.228562 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.228542 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwgcn\" (UniqueName: \"kubernetes.io/projected/92164eac-e970-48e0-88bf-5bbcb6554bef-kube-api-access-mwgcn\") pod \"multus-additional-cni-plugins-qc8kc\" (UID: \"92164eac-e970-48e0-88bf-5bbcb6554bef\") " pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.229500 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.229483 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pct8\" (UniqueName: \"kubernetes.io/projected/72260f98-ae30-4e5d-ac3f-989e32672507-kube-api-access-6pct8\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:58.300374 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.300345 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:08:58.306006 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.305988 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pcqw6" Jun 11 16:08:58.312144 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:58.312118 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce8d71cd_2b84_44af_8bb1_bf8f0daafe26.slice/crio-76a0a4aaaadb6b7b2610a0e742b9dde6a4d07d2508699d4b7fcb7c0d96f18347 WatchSource:0}: Error finding container 76a0a4aaaadb6b7b2610a0e742b9dde6a4d07d2508699d4b7fcb7c0d96f18347: Status 404 returned error can't find the container with id 76a0a4aaaadb6b7b2610a0e742b9dde6a4d07d2508699d4b7fcb7c0d96f18347 Jun 11 16:08:58.313340 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.313318 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" Jun 11 16:08:58.318990 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.318967 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-275rc" Jun 11 16:08:58.324007 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.323990 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7cb45" Jun 11 16:08:58.330961 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.330915 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:08:58.336840 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.336820 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" Jun 11 16:08:58.343084 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.343054 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8n7m9" Jun 11 16:08:58.349017 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.348999 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" Jun 11 16:08:58.713902 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.713861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:08:58.714071 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.713953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:58.714071 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:58.713988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:58.714071 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714010 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:58.714071 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714033 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:58.714071 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714043 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:58.714330 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714081 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:58.714330 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714091 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:08:59.714076495 +0000 UTC m=+4.079430645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:58.714330 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714093 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:58.714330 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714131 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:59.714114243 +0000 UTC m=+4.079468393 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:58.714330 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:58.714148 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:59.71413921 +0000 UTC m=+4.079493361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:59.060920 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.060886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:08:59.061428 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.061025 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:08:59.070701 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.070662 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"76a0a4aaaadb6b7b2610a0e742b9dde6a4d07d2508699d4b7fcb7c0d96f18347"} Jun 11 16:08:59.072872 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.072842 2560 generic.go:356] "Generic (PLEG): container finished" podID="1b99e1789793b4e1d0327783c4d1e3cb" containerID="053fda6b369939055a0ebfb6ddf9f16d4a93739ea0d3defcaab5a5b45357764c" exitCode=0 Jun 11 16:08:59.073010 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.072883 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" event={"ID":"1b99e1789793b4e1d0327783c4d1e3cb","Type":"ContainerDied","Data":"053fda6b369939055a0ebfb6ddf9f16d4a93739ea0d3defcaab5a5b45357764c"} Jun 11 16:08:59.095747 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.095710 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 16:03:58 +0000 UTC" deadline="2028-01-16 23:58:36.429118442 +0000 UTC" Jun 11 16:08:59.095747 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.095747 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14023h49m37.333377326s" Jun 11 16:08:59.128110 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:59.128080 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92164eac_e970_48e0_88bf_5bbcb6554bef.slice/crio-903247c34b9414098d7a6cc6b5d8b8ecbcb3abebf48344098ea442ffce3bf0c2 WatchSource:0}: Error finding container 903247c34b9414098d7a6cc6b5d8b8ecbcb3abebf48344098ea442ffce3bf0c2: Status 404 returned error can't find the container with id 903247c34b9414098d7a6cc6b5d8b8ecbcb3abebf48344098ea442ffce3bf0c2 Jun 11 16:08:59.128461 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:59.128436 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d61fb5d_c4b4_48db_b128_6c3ddd3f9f2e.slice/crio-7b9887276bb5032bc61552387e92bf7cca2b0013e3e07b2745e61eba0c316f83 WatchSource:0}: Error finding container 7b9887276bb5032bc61552387e92bf7cca2b0013e3e07b2745e61eba0c316f83: Status 404 returned error can't find the container with id 7b9887276bb5032bc61552387e92bf7cca2b0013e3e07b2745e61eba0c316f83 Jun 11 16:08:59.133943 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:59.133882 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad8f72cc_af12_44c9_9715_c07563601aee.slice/crio-6a900417cb7d632b65988f3a135123f79c0e9b61b8affe6b1da1f88d0a6635ef WatchSource:0}: Error finding container 6a900417cb7d632b65988f3a135123f79c0e9b61b8affe6b1da1f88d0a6635ef: Status 404 returned error can't find the container with id 6a900417cb7d632b65988f3a135123f79c0e9b61b8affe6b1da1f88d0a6635ef Jun 11 16:08:59.136629 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:08:59.136576 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77b6fa8_fa0a_4967_96fb_f1aa7a2b9ca1.slice/crio-7447324de8db083c290f97690ec030c28876325e4f967802ffe79378e68c01d0 WatchSource:0}: Error finding container 7447324de8db083c290f97690ec030c28876325e4f967802ffe79378e68c01d0: Status 404 returned error can't find the container with id 7447324de8db083c290f97690ec030c28876325e4f967802ffe79378e68c01d0 Jun 11 16:08:59.480830 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.480551 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:59.722042 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.722007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:08:59.722260 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.722071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:08:59.722260 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:08:59.722101 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:08:59.722260 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722210 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:59.722260 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722228 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:59.722260 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722238 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:59.722260 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722251 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:59.722655 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722286 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:01.722266985 +0000 UTC m=+6.087621137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:59.722655 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722210 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:59.722655 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722304 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:09:01.722294664 +0000 UTC m=+6.087648814 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:59.722655 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:08:59.722344 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:01.722325615 +0000 UTC m=+6.087679780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:00.063225 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.063191 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:00.063684 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:00.063327 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:00.063684 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.063408 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:00.063684 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:00.063480 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:00.082207 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.081477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" event={"ID":"1b99e1789793b4e1d0327783c4d1e3cb","Type":"ContainerStarted","Data":"92fb0fe9ac1e85e1639b06e680babd8f2acd7b7d779f1bc71962f77b6177cd5b"} Jun 11 16:09:00.089870 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.089805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7cb45" event={"ID":"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e","Type":"ContainerStarted","Data":"7b9887276bb5032bc61552387e92bf7cca2b0013e3e07b2745e61eba0c316f83"} Jun 11 16:09:00.096210 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.096138 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 16:03:58 +0000 UTC" deadline="2028-01-15 23:10:02.898623432 +0000 UTC" Jun 11 16:09:00.096210 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.096184 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13999h1m2.802443239s" Jun 11 16:09:00.099292 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.098756 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" event={"ID":"62b70f08a0210789f3a7a3206903da55","Type":"ContainerStarted","Data":"f95ee52ded0413df08b1bbc1f7a1d584bef7ee5929d9ba1818cf9d5b6f8fb802"} Jun 11 16:09:00.109137 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.109098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8n7m9" event={"ID":"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f","Type":"ContainerStarted","Data":"a2a3b65824972a7ed1c3dc91fc0511dc0f7a7f45fb5e8153f66ac6c5ecec0016"} Jun 11 16:09:00.111675 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.111648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-275rc" event={"ID":"ad8f72cc-af12-44c9-9715-c07563601aee","Type":"ContainerStarted","Data":"6a900417cb7d632b65988f3a135123f79c0e9b61b8affe6b1da1f88d0a6635ef"} Jun 11 16:09:00.114421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.114396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" event={"ID":"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1","Type":"ContainerStarted","Data":"7447324de8db083c290f97690ec030c28876325e4f967802ffe79378e68c01d0"} Jun 11 16:09:00.121415 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.121384 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mvs4q" event={"ID":"89d25600-3147-44e1-81e3-46f44c27e2ba","Type":"ContainerStarted","Data":"b9866d890dfc30bab228b17198026c11faccee73aa960910739925ebf6ce91b4"} Jun 11 16:09:00.124779 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.124728 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-161.ec2.internal" podStartSLOduration=3.124703347 podStartE2EDuration="3.124703347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:00.101602241 +0000 UTC m=+4.466956414" watchObservedRunningTime="2026-06-11 16:09:00.124703347 +0000 UTC m=+4.490057520" Jun 11 16:09:00.126170 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.125834 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerStarted","Data":"903247c34b9414098d7a6cc6b5d8b8ecbcb3abebf48344098ea442ffce3bf0c2"} Jun 11 16:09:00.136379 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.136352 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" event={"ID":"28a1ea03-adc1-427e-9aa8-f476cae3424b","Type":"ContainerStarted","Data":"09dfc2b04c3ccf9623d0e56dfefdab34a118f3d000a0e5db9ab6318132831e11"} Jun 11 16:09:00.141897 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:00.141874 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pcqw6" event={"ID":"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf","Type":"ContainerStarted","Data":"fe00f37b368cea06f969be6cf9522c14ac2d4c3aa036f442e8f68226ccd77f36"} Jun 11 16:09:01.060958 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:01.060637 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:01.060958 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.060770 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:01.740984 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:01.740948 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:01.741000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741050 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:01.741085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741125 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:05.741105191 +0000 UTC m=+10.106459347 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741224 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741241 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741254 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741301 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:09:05.741286247 +0000 UTC m=+10.106640403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741376 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:01.741491 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:01.741405 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:05.741397207 +0000 UTC m=+10.106751358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:02.061227 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:02.061057 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:02.061227 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:02.061206 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:02.061454 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:02.061277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:02.061454 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:02.061381 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:03.060736 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:03.060705 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:03.061230 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:03.060834 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:04.062692 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:04.062074 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:04.062692 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:04.062117 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:04.062692 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:04.062212 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:04.062692 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:04.062641 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:05.059980 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:05.059946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:05.060177 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.060081 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:05.776192 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:05.776259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:05.776289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776375 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776384 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776402 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776405 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776417 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776453 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:13.776433031 +0000 UTC m=+18.141787185 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:05.776463 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776474 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:13.776463453 +0000 UTC m=+18.141817607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:05.777366 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:05.776488 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:09:13.776480354 +0000 UTC m=+18.141834503 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:06.064966 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:06.064885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:06.065098 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:06.065014 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:06.066230 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:06.065329 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:06.066230 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:06.065413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:07.060597 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:07.060542 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:07.061179 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:07.060667 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:08.060236 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:08.060201 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:08.060395 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:08.060250 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:08.060395 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:08.060351 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:08.060523 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:08.060455 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:09.060826 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:09.060331 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:09.060826 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:09.060464 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:10.063054 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:10.063016 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:10.063482 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:10.063147 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:10.063552 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:10.063518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:10.063639 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:10.063617 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:11.060796 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:11.060757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:11.060965 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:11.060885 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:12.063520 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:12.063491 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:12.063982 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:12.063604 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:12.064074 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:12.064025 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:12.064179 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:12.064159 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:13.060481 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:13.060442 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:13.060683 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.060546 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:13.845870 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:13.845833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:13.845895 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:13.845940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.845989 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846009 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846007 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846017 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846030 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846072 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:09:29.846051736 +0000 UTC m=+34.211405888 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846089 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:29.84608177 +0000 UTC m=+34.211435919 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:13.846298 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:13.846100 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:29.846093771 +0000 UTC m=+34.211447920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:14.060504 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:14.060466 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:14.060682 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:14.060596 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:14.060905 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:14.060886 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:14.060997 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:14.060980 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:15.060543 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:15.060511 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:15.061073 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:15.060622 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:16.060750 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:16.060724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:16.061286 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:16.060787 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:16.061286 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:16.060849 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:16.061286 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:16.060963 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:17.059821 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:17.059803 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:17.059897 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:17.059881 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:18.060917 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.060625 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:18.061316 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.060682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:18.061316 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:18.061023 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:18.061316 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:18.061120 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:18.147969 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.147924 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 16:09:18.176621 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.176591 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-275rc" event={"ID":"ad8f72cc-af12-44c9-9715-c07563601aee","Type":"ContainerStarted","Data":"e0bf28d06cea8cf4bb388dc8f18ea485b1e27a49e2df6a8118753221a2f33a01"} Jun 11 16:09:18.177965 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.177940 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" event={"ID":"e77b6fa8-fa0a-4967-96fb-f1aa7a2b9ca1","Type":"ContainerStarted","Data":"eba72a2fc6423693ee847ade0c24f31e646f14fe8b6120dd8bdfd59b9770642b"} Jun 11 16:09:18.179253 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.179232 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mvs4q" event={"ID":"89d25600-3147-44e1-81e3-46f44c27e2ba","Type":"ContainerStarted","Data":"d8a75c9a5138371d8919080001277a88ab2f92ef336d952c3a5b8d893f3f0b93"} Jun 11 16:09:18.180599 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.180579 2560 generic.go:356] "Generic (PLEG): container finished" podID="92164eac-e970-48e0-88bf-5bbcb6554bef" containerID="e1d5b72df8fc1f4e622749137d3a70a9dba52185fe0a87d1a3d80d603541c1ec" exitCode=0 Jun 11 16:09:18.180671 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.180645 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerDied","Data":"e1d5b72df8fc1f4e622749137d3a70a9dba52185fe0a87d1a3d80d603541c1ec"} Jun 11 16:09:18.182463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.182441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" event={"ID":"28a1ea03-adc1-427e-9aa8-f476cae3424b","Type":"ContainerStarted","Data":"786503fbb68ee5ed8c1f0df9ac7d631e7f649b9180d455d5f22836fd52da1fbf"} Jun 11 16:09:18.182534 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.182477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" event={"ID":"28a1ea03-adc1-427e-9aa8-f476cae3424b","Type":"ContainerStarted","Data":"c461c4a089c885956da069bc18c96191a4e93e814ec57c9bf3a072a8b1020404"} Jun 11 16:09:18.183913 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.183892 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pcqw6" event={"ID":"03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf","Type":"ContainerStarted","Data":"791e7a84d07db66e2681fd568bcf230d7d2d251909999bc099af4d20ec48d9e0"} Jun 11 16:09:18.186700 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.186675 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"b19ff97cc235b5f392a80682bbf4f711451365da72a3de1365bbb88a5251c7b5"} Jun 11 16:09:18.186784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.186748 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"f0afe114afa8a1ffdb4b3f17f845690b9d719d9dd22fc06b8b15f003f629742e"} Jun 11 16:09:18.186831 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.186800 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"6a75baf6f56a7baf672e9d9bc8214a5e54f3ff5f44997b044d6ae8e305a540bc"} Jun 11 16:09:18.186877 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.186850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"bc57582f9371dc05e28357a39ded027ecd015d7601549a9919ef5a0c5be0968d"} Jun 11 16:09:18.186909 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.186897 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"13e68519d8398f1e7f707c4dce9977f18ff11eb1bb8fea206b8ab974e895ed47"} Jun 11 16:09:18.186967 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.186953 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"9d5bcb852b7f031ca6627a1da61a745dc4ebe06112bca20fc3e530e40e5b7f0b"} Jun 11 16:09:18.188048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.188026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7cb45" event={"ID":"2d61fb5d-c4b4-48db-b128-6c3ddd3f9f2e","Type":"ContainerStarted","Data":"a96286b04ed2c3825c1fe2ade4b63c07c1211a6a83a32f8680ff179b7fe41ced"} Jun 11 16:09:18.203393 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.203351 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-161.ec2.internal" podStartSLOduration=21.203338891 podStartE2EDuration="21.203338891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:00.125826877 +0000 UTC m=+4.491181048" watchObservedRunningTime="2026-06-11 16:09:18.203338891 +0000 UTC m=+22.568693063" Jun 11 16:09:18.233557 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.233497 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-275rc" podStartSLOduration=4.322319146 podStartE2EDuration="22.233486197s" totalImagesPullingTime="17.911167051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:18.205826003 +0000 UTC m=+22.571180175" watchObservedRunningTime="2026-06-11 16:09:18.233486197 +0000 UTC m=+22.598840416" Jun 11 16:09:18.279348 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.279300 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8t8k5" podStartSLOduration=4.376515455 podStartE2EDuration="22.279286005s" totalImagesPullingTime="17.90277055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:18.237187592 +0000 UTC m=+22.602541765" watchObservedRunningTime="2026-06-11 16:09:18.279286005 +0000 UTC m=+22.644640176" Jun 11 16:09:18.336987 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.336947 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7cb45" podStartSLOduration=4.407784014 podStartE2EDuration="22.336921121s" totalImagesPullingTime="17.929137107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:18.313135472 +0000 UTC m=+22.678489645" watchObservedRunningTime="2026-06-11 16:09:18.336921121 +0000 UTC m=+22.702275292" Jun 11 16:09:18.337256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:18.337231 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mvs4q" podStartSLOduration=4.43331259 podStartE2EDuration="22.337223138s" totalImagesPullingTime="17.903910548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:18.337006016 +0000 UTC m=+22.702360201" watchObservedRunningTime="2026-06-11 16:09:18.337223138 +0000 UTC m=+22.702577308" Jun 11 16:09:19.060793 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.060764 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:19.060914 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:19.060893 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:19.065486 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.065392 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T16:09:18.14796427Z","UUID":"d7e64b53-c04d-49fc-aad9-e8ebc0afcdbb","Handler":null,"Name":"","Endpoint":""} Jun 11 16:09:19.067246 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.067225 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 16:09:19.067322 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.067274 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 16:09:19.193244 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.193210 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" event={"ID":"28a1ea03-adc1-427e-9aa8-f476cae3424b","Type":"ContainerStarted","Data":"9ec07b3b3c0426e02f4b9ca03fa4349414897db9f9855ddedcda0da46ef6c65f"} Jun 11 16:09:19.195058 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.194913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8n7m9" event={"ID":"cfcca9f7-3e2f-452e-b19a-4e311fd1e05f","Type":"ContainerStarted","Data":"b182c010eb70f55bc675192098d9eb094ec184217086ae7cd789fb9fd5b1a66a"} Jun 11 16:09:19.225568 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.225525 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pcqw6" podStartSLOduration=5.367324166 podStartE2EDuration="23.225504869s" totalImagesPullingTime="17.858180703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:18.377859348 +0000 UTC m=+22.743213520" watchObservedRunningTime="2026-06-11 16:09:19.225504869 +0000 UTC m=+23.590859045" Jun 11 16:09:19.249364 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:19.249325 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59jvg" podStartSLOduration=3.796583954 podStartE2EDuration="23.249305867s" totalImagesPullingTime="19.452721913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:19.225640354 +0000 UTC m=+23.590994528" watchObservedRunningTime="2026-06-11 16:09:19.249305867 +0000 UTC m=+23.614660041" Jun 11 16:09:20.001965 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:20.001908 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:09:20.002642 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:20.002611 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:09:20.030095 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:20.030032 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8n7m9" podStartSLOduration=6.135239099 podStartE2EDuration="24.030015244s" totalImagesPullingTime="17.894776145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:19.249569991 +0000 UTC m=+23.614924164" watchObservedRunningTime="2026-06-11 16:09:20.030015244 +0000 UTC m=+24.395369415" Jun 11 16:09:20.060964 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:20.060875 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:20.061184 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:20.060875 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:20.061184 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:20.061029 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:20.061184 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:20.061159 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:20.201615 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:20.201564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"41165d068830da5be59512de4d9123eedacff7cbfd39068f03e9161d028f067d"} Jun 11 16:09:21.059917 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:21.059882 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:21.060096 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:21.060020 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:21.203476 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:21.203452 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 16:09:22.060569 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.060541 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:22.060737 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:22.060654 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:22.061081 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.061061 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:22.061209 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:22.061148 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:22.212049 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.211640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" event={"ID":"ce8d71cd-2b84-44af-8bb1-bf8f0daafe26","Type":"ContainerStarted","Data":"addee934b08914365a45f5d707adc0eca52bff77442fc42a218dfce0ebd5167a"} Jun 11 16:09:22.212323 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.212295 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:09:22.212530 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.212490 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:09:22.212664 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.212642 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:09:22.235729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.235077 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:09:22.236892 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.236509 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:09:22.249350 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:22.249313 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" podStartSLOduration=7.159503338 podStartE2EDuration="26.249299578s" totalImagesPullingTime="19.08979624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:22.24586907 +0000 UTC m=+26.611223243" watchObservedRunningTime="2026-06-11 16:09:22.249299578 +0000 UTC m=+26.614653749" Jun 11 16:09:23.060304 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:23.060279 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:23.060439 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:23.060367 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:23.215743 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:23.215706 2560 generic.go:356] "Generic (PLEG): container finished" podID="92164eac-e970-48e0-88bf-5bbcb6554bef" containerID="92bc9038dbfb7cfc9c3459542150c07e6b2efd73ebe4c36ce01f58fc235c3dfc" exitCode=0 Jun 11 16:09:23.216183 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:23.215801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerDied","Data":"92bc9038dbfb7cfc9c3459542150c07e6b2efd73ebe4c36ce01f58fc235c3dfc"} Jun 11 16:09:24.060945 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.060758 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:24.061104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.060813 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:24.061104 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:24.061047 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:24.061182 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:24.061097 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:24.262730 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.262514 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xbj5v"] Jun 11 16:09:24.262730 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.262622 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:24.263287 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:24.262729 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:24.266151 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.266124 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-77sz2"] Jun 11 16:09:24.266259 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.266226 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:24.266352 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:24.266329 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:24.266690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.266667 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gjz8k"] Jun 11 16:09:24.266806 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:24.266757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:24.266873 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:24.266840 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:25.222038 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:25.222002 2560 generic.go:356] "Generic (PLEG): container finished" podID="92164eac-e970-48e0-88bf-5bbcb6554bef" containerID="626cd3fdc0a091bf8d2849cf24d0b67e9694d8735bdb250da27d4fc57c713beb" exitCode=0 Jun 11 16:09:25.222209 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:25.222048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerDied","Data":"626cd3fdc0a091bf8d2849cf24d0b67e9694d8735bdb250da27d4fc57c713beb"} Jun 11 16:09:26.063257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:26.063225 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:26.063791 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:26.063370 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:26.064243 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:26.064221 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:26.065567 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:26.064342 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:26.065567 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:26.064535 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:26.065567 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:26.064689 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:27.228575 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:27.228368 2560 generic.go:356] "Generic (PLEG): container finished" podID="92164eac-e970-48e0-88bf-5bbcb6554bef" containerID="cad43508f9adc98965a09a35a7216ade5816eba842f6b6e80279e062d8e0cd94" exitCode=0 Jun 11 16:09:27.229041 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:27.228441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerDied","Data":"cad43508f9adc98965a09a35a7216ade5816eba842f6b6e80279e062d8e0cd94"} Jun 11 16:09:27.959895 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:27.959857 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:09:27.960089 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:27.959998 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 11 16:09:27.960709 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:27.960650 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mvs4q" Jun 11 16:09:28.060142 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:28.060114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:28.060339 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:28.060151 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:28.060339 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:28.060114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:28.060339 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:28.060248 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:09:28.060478 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:28.060332 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbj5v" podUID="0ff4b19f-3067-44b8-bf80-4a7041a91297" Jun 11 16:09:28.060478 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:28.060410 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gjz8k" podUID="1d296f9d-42f2-4021-ab18-6931fcfbbfed" Jun 11 16:09:29.865674 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:29.865617 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:29.865685 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:29.865719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865781 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865792 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865811 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865823 2560 projected.go:196] Error preparing data for projected volume kube-api-access-pcq4k for pod openshift-network-diagnostics/network-check-target-gjz8k: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865841 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865859 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret podName:0ff4b19f-3067-44b8-bf80-4a7041a91297 nodeName:}" failed. No retries permitted until 2026-06-11 16:10:01.865839189 +0000 UTC m=+66.231193342 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret") pod "global-pull-secret-syncer-xbj5v" (UID: "0ff4b19f-3067-44b8-bf80-4a7041a91297") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865874 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k podName:1d296f9d-42f2-4021-ab18-6931fcfbbfed nodeName:}" failed. No retries permitted until 2026-06-11 16:10:01.865867794 +0000 UTC m=+66.231221944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pcq4k" (UniqueName: "kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k") pod "network-check-target-gjz8k" (UID: "1d296f9d-42f2-4021-ab18-6931fcfbbfed") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:29.866093 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:29.865896 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:10:01.865878856 +0000 UTC m=+66.231233022 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:29.970733 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:29.970699 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-161.ec2.internal" event="NodeReady" Jun 11 16:09:29.970911 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:29.970857 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 16:09:30.032003 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.031976 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7bmts"] Jun 11 16:09:30.065175 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.065145 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-z2szh"] Jun 11 16:09:30.065337 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.065220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:09:30.065337 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.065243 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:09:30.065417 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.065220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:09:30.065417 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.065367 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.067924 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.067860 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 16:09:30.068088 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.068012 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 16:09:30.068164 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.068106 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kx8mw\"" Jun 11 16:09:30.068391 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.068373 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2w2xq\"" Jun 11 16:09:30.069735 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.069716 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 16:09:30.070036 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.070018 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 16:09:30.070127 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.070050 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 16:09:30.070252 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.070239 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-phqs4\"" Jun 11 16:09:30.070432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.070412 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 16:09:30.071629 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.071613 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 16:09:30.085664 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.085636 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7bmts"] Jun 11 16:09:30.085760 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.085677 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z2szh"] Jun 11 16:09:30.085963 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.085890 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.088705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.088688 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 16:09:30.088705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.088695 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 16:09:30.088961 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.088943 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pbxmp\"" Jun 11 16:09:30.167611 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.167571 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73dcc6de-fa1c-4145-a113-653c0863671e-cert\") pod \"ingress-canary-7bmts\" (UID: \"73dcc6de-fa1c-4145-a113-653c0863671e\") " pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.167770 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.167673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf5sq\" (UniqueName: \"kubernetes.io/projected/73dcc6de-fa1c-4145-a113-653c0863671e-kube-api-access-kf5sq\") pod \"ingress-canary-7bmts\" (UID: \"73dcc6de-fa1c-4145-a113-653c0863671e\") " pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.268633 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.268599 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws858\" (UniqueName: \"kubernetes.io/projected/09604985-6b11-49bc-98d6-1331719d4f16-kube-api-access-ws858\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.268792 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.268668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73dcc6de-fa1c-4145-a113-653c0863671e-cert\") pod \"ingress-canary-7bmts\" (UID: \"73dcc6de-fa1c-4145-a113-653c0863671e\") " pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.268792 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.268715 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09604985-6b11-49bc-98d6-1331719d4f16-tmp-dir\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.268792 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.268745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.268792 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.268786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/09604985-6b11-49bc-98d6-1331719d4f16-config-volume\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.268968 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.268883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf5sq\" (UniqueName: \"kubernetes.io/projected/73dcc6de-fa1c-4145-a113-653c0863671e-kube-api-access-kf5sq\") pod \"ingress-canary-7bmts\" (UID: \"73dcc6de-fa1c-4145-a113-653c0863671e\") " pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.273978 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.273954 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/73dcc6de-fa1c-4145-a113-653c0863671e-cert\") pod \"ingress-canary-7bmts\" (UID: \"73dcc6de-fa1c-4145-a113-653c0863671e\") " pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.278890 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.278869 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf5sq\" (UniqueName: \"kubernetes.io/projected/73dcc6de-fa1c-4145-a113-653c0863671e-kube-api-access-kf5sq\") pod \"ingress-canary-7bmts\" (UID: \"73dcc6de-fa1c-4145-a113-653c0863671e\") " pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.369499 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.369414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws858\" (UniqueName: \"kubernetes.io/projected/09604985-6b11-49bc-98d6-1331719d4f16-kube-api-access-ws858\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.369499 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.369490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09604985-6b11-49bc-98d6-1331719d4f16-tmp-dir\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.369706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.369519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.369706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.369563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/09604985-6b11-49bc-98d6-1331719d4f16-config-volume\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.369706 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:30.369666 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:30.369852 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:30.369736 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:30.86971296 +0000 UTC m=+35.235067112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:09:30.369945 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.369908 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09604985-6b11-49bc-98d6-1331719d4f16-tmp-dir\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.370141 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.370124 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/09604985-6b11-49bc-98d6-1331719d4f16-config-volume\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.385001 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.384973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws858\" (UniqueName: \"kubernetes.io/projected/09604985-6b11-49bc-98d6-1331719d4f16-kube-api-access-ws858\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.397923 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.397902 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7bmts" Jun 11 16:09:30.873380 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:30.873341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:30.873856 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:30.873513 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:30.873856 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:30.873590 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:31.87356975 +0000 UTC m=+36.238923902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:09:31.459607 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:31.459575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7bmts"] Jun 11 16:09:31.466253 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:09:31.466224 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73dcc6de_fa1c_4145_a113_653c0863671e.slice/crio-f380593b6ef13d70d01b1d47ba229055626046e1dcc490ee482da2effc8001f4 WatchSource:0}: Error finding container f380593b6ef13d70d01b1d47ba229055626046e1dcc490ee482da2effc8001f4: Status 404 returned error can't find the container with id f380593b6ef13d70d01b1d47ba229055626046e1dcc490ee482da2effc8001f4 Jun 11 16:09:31.879541 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:31.879515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:31.879874 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:31.879660 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:31.879874 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:31.879715 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:33.879699998 +0000 UTC m=+38.245054148 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:09:32.244084 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:32.243849 2560 generic.go:356] "Generic (PLEG): container finished" podID="92164eac-e970-48e0-88bf-5bbcb6554bef" containerID="c52805bdc16ce0343f4bc76b77c999946085357913d8ebc12696983295fe2a48" exitCode=0 Jun 11 16:09:32.244260 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:32.243963 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerDied","Data":"c52805bdc16ce0343f4bc76b77c999946085357913d8ebc12696983295fe2a48"} Jun 11 16:09:32.245554 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:32.245529 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7bmts" event={"ID":"73dcc6de-fa1c-4145-a113-653c0863671e","Type":"ContainerStarted","Data":"f380593b6ef13d70d01b1d47ba229055626046e1dcc490ee482da2effc8001f4"} Jun 11 16:09:33.252964 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:33.252769 2560 generic.go:356] "Generic (PLEG): container finished" podID="92164eac-e970-48e0-88bf-5bbcb6554bef" containerID="245cff80909478a046d2a1688c3ef088858a7d7bda2102d6ebe55dd6375c1c96" exitCode=0 Jun 11 16:09:33.252964 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:33.252836 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerDied","Data":"245cff80909478a046d2a1688c3ef088858a7d7bda2102d6ebe55dd6375c1c96"} Jun 11 16:09:33.894955 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:33.894914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:33.895086 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:33.895064 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:33.895137 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:33.895127 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:37.895111909 +0000 UTC m=+42.260466059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:09:34.258657 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:34.258625 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" event={"ID":"92164eac-e970-48e0-88bf-5bbcb6554bef","Type":"ContainerStarted","Data":"0c4642f16bf954755a50d2c19940cd430774a7905bb069f87f14de1076e3f477"} Jun 11 16:09:34.260051 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:34.260026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7bmts" event={"ID":"73dcc6de-fa1c-4145-a113-653c0863671e","Type":"ContainerStarted","Data":"f611c0e7c22105d8e77e82285951398cbe55c20f96ce3f620757e49fe642b67e"} Jun 11 16:09:34.284643 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:34.284601 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qc8kc" podStartSLOduration=9.835058556 podStartE2EDuration="38.284588901s" totalImagesPullingTime="28.449530345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:34.282576337 +0000 UTC m=+38.647930509" watchObservedRunningTime="2026-06-11 16:09:34.284588901 +0000 UTC m=+38.649943073" Jun 11 16:09:34.302023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:34.301980 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7bmts" podStartSLOduration=1.936935321 podStartE2EDuration="4.301963679s" totalImagesPullingTime="2.365028358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:34.301055881 +0000 UTC m=+38.666410052" watchObservedRunningTime="2026-06-11 16:09:34.301963679 +0000 UTC m=+38.667317851" Jun 11 16:09:37.919782 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:37.919737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:37.920205 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:37.919878 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:37.920205 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:37.919960 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:45.919920928 +0000 UTC m=+50.285275079 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:09:45.970422 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:45.970384 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:09:45.970976 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:45.970498 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:45.970976 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:09:45.970556 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:10:01.970543529 +0000 UTC m=+66.335897680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:09:54.235410 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:09:54.235383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cfvzx" Jun 11 16:10:01.873904 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.873861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:10:01.873904 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.873904 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:10:01.874340 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.873961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:10:01.877049 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.877015 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 16:10:01.877125 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.877085 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 16:10:01.877460 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.877441 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 16:10:01.884996 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:10:01.884966 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:10:01.885106 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:10:01.885035 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:11:05.885021229 +0000 UTC m=+130.250375380 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : secret "metrics-daemon-secret" not found Jun 11 16:10:01.887651 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.887634 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 16:10:01.888285 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.888267 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0ff4b19f-3067-44b8-bf80-4a7041a91297-original-pull-secret\") pod \"global-pull-secret-syncer-xbj5v\" (UID: \"0ff4b19f-3067-44b8-bf80-4a7041a91297\") " pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:10:01.897573 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.897551 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcq4k\" (UniqueName: \"kubernetes.io/projected/1d296f9d-42f2-4021-ab18-6931fcfbbfed-kube-api-access-pcq4k\") pod \"network-check-target-gjz8k\" (UID: \"1d296f9d-42f2-4021-ab18-6931fcfbbfed\") " pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:10:01.975056 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:01.975025 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:10:01.975159 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:10:01.975116 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:10:01.975197 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:10:01.975173 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:10:33.975158941 +0000 UTC m=+98.340513091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:10:02.185250 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:02.185160 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbj5v" Jun 11 16:10:02.193987 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:02.193964 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2w2xq\"" Jun 11 16:10:02.202383 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:02.202363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:10:02.356599 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:02.356560 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xbj5v"] Jun 11 16:10:02.369239 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:02.369217 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gjz8k"] Jun 11 16:10:02.371721 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:10:02.371698 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d296f9d_42f2_4021_ab18_6931fcfbbfed.slice/crio-0878289baea3789f7d76e736c0a15732df6a3369fbd5376bbf2e6ca7d021441c WatchSource:0}: Error finding container 0878289baea3789f7d76e736c0a15732df6a3369fbd5376bbf2e6ca7d021441c: Status 404 returned error can't find the container with id 0878289baea3789f7d76e736c0a15732df6a3369fbd5376bbf2e6ca7d021441c Jun 11 16:10:03.325486 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:03.325442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gjz8k" event={"ID":"1d296f9d-42f2-4021-ab18-6931fcfbbfed","Type":"ContainerStarted","Data":"0878289baea3789f7d76e736c0a15732df6a3369fbd5376bbf2e6ca7d021441c"} Jun 11 16:10:03.326723 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:03.326687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xbj5v" event={"ID":"0ff4b19f-3067-44b8-bf80-4a7041a91297","Type":"ContainerStarted","Data":"d76cbefd7b077574439b7d5b6bed6a38e609fabf2f7f81555ef3740ad656f4a5"} Jun 11 16:10:05.333095 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:05.333062 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gjz8k" event={"ID":"1d296f9d-42f2-4021-ab18-6931fcfbbfed","Type":"ContainerStarted","Data":"2c0e71ec748eef05e49df253d9d273b2734265e8cb6ea5e14b509517426e5ab5"} Jun 11 16:10:05.333488 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:05.333289 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:10:05.349635 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:05.349560 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gjz8k" podStartSLOduration=66.630858378 podStartE2EDuration="1m9.349547859s" totalImagesPullingTime="2.718689481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:10:05.348251528 +0000 UTC m=+69.713605697" watchObservedRunningTime="2026-06-11 16:10:05.349547859 +0000 UTC m=+69.714902031" Jun 11 16:10:08.340713 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:08.340672 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xbj5v" event={"ID":"0ff4b19f-3067-44b8-bf80-4a7041a91297","Type":"ContainerStarted","Data":"20c71426e04562ce8b174f1c17ff9029b87fd90f0dfe840db9515d8560c3549c"} Jun 11 16:10:08.358590 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:08.358547 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xbj5v" podStartSLOduration=66.915656426 podStartE2EDuration="1m12.358533593s" totalImagesPullingTime="5.442877167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:10:08.357902381 +0000 UTC m=+72.723256557" watchObservedRunningTime="2026-06-11 16:10:08.358533593 +0000 UTC m=+72.723887765" Jun 11 16:10:33.986564 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:33.986513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:10:33.986988 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:10:33.986628 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:10:33.986988 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:10:33.986693 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:11:37.986679177 +0000 UTC m=+162.352033328 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:10:36.337839 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:10:36.337810 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gjz8k" Jun 11 16:11:05.905421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:11:05.905382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:11:05.905917 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:11:05.905498 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:11:05.905917 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:11:05.905552 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:07.905537328 +0000 UTC m=+252.270891478 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : secret "metrics-daemon-secret" not found Jun 11 16:11:33.078213 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:11:33.078174 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:11:33.104490 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:11:33.104451 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-z2szh" podUID="09604985-6b11-49bc-98d6-1331719d4f16" Jun 11 16:11:38.009846 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:11:38.009804 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:11:38.010376 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:11:38.009955 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:11:38.010376 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:11:38.010027 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls podName:09604985-6b11-49bc-98d6-1331719d4f16 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:40.010005209 +0000 UTC m=+284.375359375 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls") pod "dns-default-z2szh" (UID: "09604985-6b11-49bc-98d6-1331719d4f16") : secret "dns-default-metrics-tls" not found Jun 11 16:11:45.060831 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:11:45.060742 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2szh" Jun 11 16:11:47.060629 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:11:47.060590 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:12:59.170246 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.170212 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp"] Jun 11 16:12:59.171987 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.171968 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" Jun 11 16:12:59.176961 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.176922 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-b66bj\"" Jun 11 16:12:59.183033 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.182993 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz"] Jun 11 16:12:59.184729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.184713 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-z5kgd"] Jun 11 16:12:59.184835 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.184820 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.186443 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.186424 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.187306 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.187284 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk"] Jun 11 16:12:59.187420 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.187403 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 11 16:12:59.188971 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.188954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.189432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.189417 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-hn7hd\"" Jun 11 16:12:59.192431 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.192407 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 11 16:12:59.201123 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.201104 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 11 16:12:59.201236 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.201145 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 11 16:12:59.201316 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.201191 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.202115 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.202095 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5"] Jun 11 16:12:59.203168 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.203145 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 11 16:12:59.204167 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204149 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.204381 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204360 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp"] Jun 11 16:12:59.204783 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204675 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz"] Jun 11 16:12:59.205308 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204558 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 11 16:12:59.205308 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204628 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.205308 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204555 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.205507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.204710 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-x976k\"" Jun 11 16:12:59.205507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.205431 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9dxj7\"" Jun 11 16:12:59.205619 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.205604 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.206500 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.206482 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-z5kgd"] Jun 11 16:12:59.208077 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.208057 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.208291 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.208275 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.209459 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.209180 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pt845\"" Jun 11 16:12:59.209459 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.209422 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jun 11 16:12:59.209576 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.209562 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jun 11 16:12:59.212277 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.212259 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 11 16:12:59.215802 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.215508 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk"] Jun 11 16:12:59.223258 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.223222 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5"] Jun 11 16:12:59.233446 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.233424 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt5q4\" (UniqueName: \"kubernetes.io/projected/34dd110b-5257-42f1-9069-7c21ec59ae5d-kube-api-access-vt5q4\") pod \"network-check-source-dbd9cf844-8nfrp\" (UID: \"34dd110b-5257-42f1-9069-7c21ec59ae5d\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" Jun 11 16:12:59.274031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.273999 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr"] Jun 11 16:12:59.275671 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.275655 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz"] Jun 11 16:12:59.275810 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.275794 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" Jun 11 16:12:59.277253 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.277236 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6"] Jun 11 16:12:59.277377 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.277361 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.279029 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.279008 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.280162 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280145 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.280458 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280437 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.280458 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280447 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 11 16:12:59.280604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280486 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5jpmv\"" Jun 11 16:12:59.280604 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280522 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.280707 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280632 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-86qb6\"" Jun 11 16:12:59.280983 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.280966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 11 16:12:59.281067 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.281041 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.281413 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.281393 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.281491 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.281393 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.281491 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.281400 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nrlwl\"" Jun 11 16:12:59.281491 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.281404 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 11 16:12:59.286264 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.286245 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cf8c75967-2jd6s"] Jun 11 16:12:59.287917 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.287901 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.290566 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.290545 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 16:12:59.290566 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.290564 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 16:12:59.290725 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.290546 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 16:12:59.290725 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.290606 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2gg8w\"" Jun 11 16:12:59.301579 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.301554 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz"] Jun 11 16:12:59.302243 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.302223 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr"] Jun 11 16:12:59.306385 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.306367 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 16:12:59.313979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.313961 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cf8c75967-2jd6s"] Jun 11 16:12:59.334131 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/559452f3-f51f-44c6-8997-04f31fcae598-serving-cert\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.334242 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334134 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhd5\" (UniqueName: \"kubernetes.io/projected/79be130d-d4cc-46d4-a56c-5e216839fcc5-kube-api-access-xvhd5\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.334242 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334152 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.334242 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559452f3-f51f-44c6-8997-04f31fcae598-trusted-ca-bundle\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.334242 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9d22\" (UniqueName: \"kubernetes.io/projected/4771f439-c1d3-4e27-befd-2ddafdee240e-kube-api-access-d9d22\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.334242 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559452f3-f51f-44c6-8997-04f31fcae598-service-ca-bundle\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.334408 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m5ns\" (UniqueName: \"kubernetes.io/projected/559452f3-f51f-44c6-8997-04f31fcae598-kube-api-access-2m5ns\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.334408 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d690600-b47d-4607-a3d7-739b5525f4e4-config\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.334408 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/559452f3-f51f-44c6-8997-04f31fcae598-snapshots\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.334408 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2z48\" (UniqueName: \"kubernetes.io/projected/8d690600-b47d-4607-a3d7-739b5525f4e4-kube-api-access-f2z48\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.334408 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79be130d-d4cc-46d4-a56c-5e216839fcc5-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.334408 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334397 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559452f3-f51f-44c6-8997-04f31fcae598-tmp\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2b56d67a-a366-4ea7-be50-7f7f793cc700-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334455 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d690600-b47d-4607-a3d7-739b5525f4e4-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334504 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/209621f8-7603-4523-b0bb-d4e1f61b6913-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334532 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79be130d-d4cc-46d4-a56c-5e216839fcc5-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.334578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334563 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt5q4\" (UniqueName: \"kubernetes.io/projected/34dd110b-5257-42f1-9069-7c21ec59ae5d-kube-api-access-vt5q4\") pod \"network-check-source-dbd9cf844-8nfrp\" (UID: \"34dd110b-5257-42f1-9069-7c21ec59ae5d\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" Jun 11 16:12:59.334787 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smf7z\" (UniqueName: \"kubernetes.io/projected/c9c0c5be-ef6e-4864-b60f-889eff72ad24-kube-api-access-smf7z\") pod \"volume-data-source-validator-78dd696465-rs2xr\" (UID: \"c9c0c5be-ef6e-4864-b60f-889eff72ad24\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" Jun 11 16:12:59.334787 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.334619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdd9r\" (UniqueName: \"kubernetes.io/projected/2b56d67a-a366-4ea7-be50-7f7f793cc700-kube-api-access-wdd9r\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.348549 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.348525 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt5q4\" (UniqueName: \"kubernetes.io/projected/34dd110b-5257-42f1-9069-7c21ec59ae5d-kube-api-access-vt5q4\") pod \"network-check-source-dbd9cf844-8nfrp\" (UID: \"34dd110b-5257-42f1-9069-7c21ec59ae5d\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" Jun 11 16:12:59.353997 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.353974 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6"] Jun 11 16:12:59.428645 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.428585 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-jfsm9"] Jun 11 16:12:59.434401 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.434378 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.434958 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.434924 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/559452f3-f51f-44c6-8997-04f31fcae598-snapshots\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.435053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.434967 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z48\" (UniqueName: \"kubernetes.io/projected/8d690600-b47d-4607-a3d7-739b5525f4e4-kube-api-access-f2z48\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.435053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.434996 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-trusted-ca\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.435053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435024 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4sbq\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-kube-api-access-l4sbq\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.435053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79be130d-d4cc-46d4-a56c-5e216839fcc5-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.435259 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559452f3-f51f-44c6-8997-04f31fcae598-tmp\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.435259 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.435259 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2b56d67a-a366-4ea7-be50-7f7f793cc700-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.435259 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d690600-b47d-4607-a3d7-739b5525f4e4-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.435259 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/209621f8-7603-4523-b0bb-d4e1f61b6913-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-image-registry-private-configuration\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.435304 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435337 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79be130d-d4cc-46d4-a56c-5e216839fcc5-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/201a346a-f445-40f8-8bcf-8e5cb113cc6d-ca-trust-extracted\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.435452 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.435461 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls podName:2b56d67a-a366-4ea7-be50-7f7f793cc700 nodeName:}" failed. No retries permitted until 2026-06-11 16:12:59.93537255 +0000 UTC m=+244.300726703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-rfvb5" (UID: "2b56d67a-a366-4ea7-be50-7f7f793cc700") : secret "cluster-monitoring-operator-tls" not found Jun 11 16:12:59.435581 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435537 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559452f3-f51f-44c6-8997-04f31fcae598-tmp\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.435611 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls podName:4771f439-c1d3-4e27-befd-2ddafdee240e nodeName:}" failed. No retries permitted until 2026-06-11 16:12:59.935594897 +0000 UTC m=+244.300949050 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-df2r6" (UID: "4771f439-c1d3-4e27-befd-2ddafdee240e") : secret "samples-operator-tls" not found Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-certificates\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smf7z\" (UniqueName: \"kubernetes.io/projected/c9c0c5be-ef6e-4864-b60f-889eff72ad24-kube-api-access-smf7z\") pod \"volume-data-source-validator-78dd696465-rs2xr\" (UID: \"c9c0c5be-ef6e-4864-b60f-889eff72ad24\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435685 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/79be130d-d4cc-46d4-a56c-5e216839fcc5-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdd9r\" (UniqueName: \"kubernetes.io/projected/2b56d67a-a366-4ea7-be50-7f7f793cc700-kube-api-access-wdd9r\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435752 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-installation-pull-secrets\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/559452f3-f51f-44c6-8997-04f31fcae598-serving-cert\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhd5\" (UniqueName: \"kubernetes.io/projected/79be130d-d4cc-46d4-a56c-5e216839fcc5-kube-api-access-xvhd5\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435831 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559452f3-f51f-44c6-8997-04f31fcae598-trusted-ca-bundle\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9d22\" (UniqueName: \"kubernetes.io/projected/4771f439-c1d3-4e27-befd-2ddafdee240e-kube-api-access-d9d22\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559452f3-f51f-44c6-8997-04f31fcae598-service-ca-bundle\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.436025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.435974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m5ns\" (UniqueName: \"kubernetes.io/projected/559452f3-f51f-44c6-8997-04f31fcae598-kube-api-access-2m5ns\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.436868 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.436000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d690600-b47d-4607-a3d7-739b5525f4e4-config\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.436868 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.436025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-bound-sa-token\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.436868 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.436051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2b56d67a-a366-4ea7-be50-7f7f793cc700-telemetry-config\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.436868 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.436112 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:12:59.436868 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.436166 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert podName:209621f8-7603-4523-b0bb-d4e1f61b6913 nodeName:}" failed. No retries permitted until 2026-06-11 16:12:59.936150067 +0000 UTC m=+244.301504218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-fqlwz" (UID: "209621f8-7603-4523-b0bb-d4e1f61b6913") : secret "networking-console-plugin-cert" not found Jun 11 16:12:59.436868 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.436816 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d690600-b47d-4607-a3d7-739b5525f4e4-config\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.437204 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.436866 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559452f3-f51f-44c6-8997-04f31fcae598-service-ca-bundle\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.437204 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.437009 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/559452f3-f51f-44c6-8997-04f31fcae598-trusted-ca-bundle\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.437204 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.437153 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/559452f3-f51f-44c6-8997-04f31fcae598-snapshots\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.437716 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.437691 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/209621f8-7603-4523-b0bb-d4e1f61b6913-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.438031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.438010 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/79be130d-d4cc-46d4-a56c-5e216839fcc5-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.438225 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.438211 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d690600-b47d-4607-a3d7-739b5525f4e4-serving-cert\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.438853 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.438837 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/559452f3-f51f-44c6-8997-04f31fcae598-serving-cert\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.439710 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.439693 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.439759 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.439726 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 11 16:12:59.440176 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.440162 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.440310 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.440295 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 11 16:12:59.440377 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.440367 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p67z7\"" Jun 11 16:12:59.444821 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.444802 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 11 16:12:59.448829 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.448810 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-jfsm9"] Jun 11 16:12:59.463308 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.463282 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smf7z\" (UniqueName: \"kubernetes.io/projected/c9c0c5be-ef6e-4864-b60f-889eff72ad24-kube-api-access-smf7z\") pod \"volume-data-source-validator-78dd696465-rs2xr\" (UID: \"c9c0c5be-ef6e-4864-b60f-889eff72ad24\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" Jun 11 16:12:59.463390 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.463285 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhd5\" (UniqueName: \"kubernetes.io/projected/79be130d-d4cc-46d4-a56c-5e216839fcc5-kube-api-access-xvhd5\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-xv5fk\" (UID: \"79be130d-d4cc-46d4-a56c-5e216839fcc5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.463432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.463414 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdd9r\" (UniqueName: \"kubernetes.io/projected/2b56d67a-a366-4ea7-be50-7f7f793cc700-kube-api-access-wdd9r\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.467103 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.467083 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9d22\" (UniqueName: \"kubernetes.io/projected/4771f439-c1d3-4e27-befd-2ddafdee240e-kube-api-access-d9d22\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.470667 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.470647 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m5ns\" (UniqueName: \"kubernetes.io/projected/559452f3-f51f-44c6-8997-04f31fcae598-kube-api-access-2m5ns\") pod \"insights-operator-896549976-z5kgd\" (UID: \"559452f3-f51f-44c6-8997-04f31fcae598\") " pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.472558 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.472540 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2z48\" (UniqueName: \"kubernetes.io/projected/8d690600-b47d-4607-a3d7-739b5525f4e4-kube-api-access-f2z48\") pod \"service-ca-operator-6bc5ff68f5-zkhcz\" (UID: \"8d690600-b47d-4607-a3d7-739b5525f4e4\") " pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.480292 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.480274 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" Jun 11 16:12:59.505309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.505287 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-z5kgd" Jun 11 16:12:59.514768 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.514713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" Jun 11 16:12:59.537425 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537391 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8f7w\" (UniqueName: \"kubernetes.io/projected/9a3095c8-d355-4890-b985-846e3bb2e3e4-kube-api-access-p8f7w\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.537547 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-trusted-ca\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.537547 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4sbq\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-kube-api-access-l4sbq\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.537650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537565 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9a3095c8-d355-4890-b985-846e3bb2e3e4-trusted-ca\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.537650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537594 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a3095c8-d355-4890-b985-846e3bb2e3e4-config\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.537751 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537649 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a3095c8-d355-4890-b985-846e3bb2e3e4-serving-cert\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.537751 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-image-registry-private-configuration\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.537979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537956 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/201a346a-f445-40f8-8bcf-8e5cb113cc6d-ca-trust-extracted\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.538047 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.537999 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-certificates\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.538104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.538046 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.538104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.538074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-installation-pull-secrets\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.538193 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.538142 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-bound-sa-token\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.538705 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.538380 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:12:59.538705 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.538407 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cf8c75967-2jd6s: secret "image-registry-tls" not found Jun 11 16:12:59.538705 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.538490 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls podName:201a346a-f445-40f8-8bcf-8e5cb113cc6d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:00.038467469 +0000 UTC m=+244.403821670 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls") pod "image-registry-7cf8c75967-2jd6s" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d") : secret "image-registry-tls" not found Jun 11 16:12:59.538705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.538562 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-trusted-ca\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.538705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.538570 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-certificates\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.539036 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.538839 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/201a346a-f445-40f8-8bcf-8e5cb113cc6d-ca-trust-extracted\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.542031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.541652 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-image-registry-private-configuration\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.542031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.541992 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-installation-pull-secrets\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.556778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.556752 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-bound-sa-token\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.561511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.561455 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4sbq\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-kube-api-access-l4sbq\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:12:59.585295 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.585265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" Jun 11 16:12:59.593494 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.593398 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" Jun 11 16:12:59.640834 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.640148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8f7w\" (UniqueName: \"kubernetes.io/projected/9a3095c8-d355-4890-b985-846e3bb2e3e4-kube-api-access-p8f7w\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.640834 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.640216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9a3095c8-d355-4890-b985-846e3bb2e3e4-trusted-ca\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.640834 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.640246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a3095c8-d355-4890-b985-846e3bb2e3e4-config\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.640834 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.640300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a3095c8-d355-4890-b985-846e3bb2e3e4-serving-cert\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.642603 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.642344 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9a3095c8-d355-4890-b985-846e3bb2e3e4-config\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.642603 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.642597 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9a3095c8-d355-4890-b985-846e3bb2e3e4-trusted-ca\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.646949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.646856 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp"] Jun 11 16:12:59.647326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.647283 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9a3095c8-d355-4890-b985-846e3bb2e3e4-serving-cert\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.649205 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:12:59.649182 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34dd110b_5257_42f1_9069_7c21ec59ae5d.slice/crio-b6375ea94d0c414d4a4b069ae740da4cc26c0fffbec5347c0a990fbf19106755 WatchSource:0}: Error finding container b6375ea94d0c414d4a4b069ae740da4cc26c0fffbec5347c0a990fbf19106755: Status 404 returned error can't find the container with id b6375ea94d0c414d4a4b069ae740da4cc26c0fffbec5347c0a990fbf19106755 Jun 11 16:12:59.660034 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.659612 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8f7w\" (UniqueName: \"kubernetes.io/projected/9a3095c8-d355-4890-b985-846e3bb2e3e4-kube-api-access-p8f7w\") pod \"console-operator-668fdbf85c-jfsm9\" (UID: \"9a3095c8-d355-4890-b985-846e3bb2e3e4\") " pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.676845 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.676803 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-z5kgd"] Jun 11 16:12:59.686151 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:12:59.686124 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559452f3_f51f_44c6_8997_04f31fcae598.slice/crio-dbd64e4eff537e67dfe2430cc130e603616b4a50eabf1b80bff689691d442deb WatchSource:0}: Error finding container dbd64e4eff537e67dfe2430cc130e603616b4a50eabf1b80bff689691d442deb: Status 404 returned error can't find the container with id dbd64e4eff537e67dfe2430cc130e603616b4a50eabf1b80bff689691d442deb Jun 11 16:12:59.701655 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.701631 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk"] Jun 11 16:12:59.711242 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:12:59.711211 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79be130d_d4cc_46d4_a56c_5e216839fcc5.slice/crio-21eca320008f7c67dc48213cc4960bc105914f5aa6c6d1d5c470276ecff40ab6 WatchSource:0}: Error finding container 21eca320008f7c67dc48213cc4960bc105914f5aa6c6d1d5c470276ecff40ab6: Status 404 returned error can't find the container with id 21eca320008f7c67dc48213cc4960bc105914f5aa6c6d1d5c470276ecff40ab6 Jun 11 16:12:59.737615 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.737587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" event={"ID":"79be130d-d4cc-46d4-a56c-5e216839fcc5","Type":"ContainerStarted","Data":"21eca320008f7c67dc48213cc4960bc105914f5aa6c6d1d5c470276ecff40ab6"} Jun 11 16:12:59.738770 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.738746 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-z5kgd" event={"ID":"559452f3-f51f-44c6-8997-04f31fcae598","Type":"ContainerStarted","Data":"dbd64e4eff537e67dfe2430cc130e603616b4a50eabf1b80bff689691d442deb"} Jun 11 16:12:59.740243 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.740224 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" event={"ID":"34dd110b-5257-42f1-9069-7c21ec59ae5d","Type":"ContainerStarted","Data":"b6375ea94d0c414d4a4b069ae740da4cc26c0fffbec5347c0a990fbf19106755"} Jun 11 16:12:59.754110 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.754087 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:12:59.758483 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.758460 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr"] Jun 11 16:12:59.763720 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:12:59.763619 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9c0c5be_ef6e_4864_b60f_889eff72ad24.slice/crio-8875383c5fc67debc1e2a3984be2ff56fac72eb80492a4c86a6b97da5521858a WatchSource:0}: Error finding container 8875383c5fc67debc1e2a3984be2ff56fac72eb80492a4c86a6b97da5521858a: Status 404 returned error can't find the container with id 8875383c5fc67debc1e2a3984be2ff56fac72eb80492a4c86a6b97da5521858a Jun 11 16:12:59.788616 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.788587 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz"] Jun 11 16:12:59.789906 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:12:59.789877 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d690600_b47d_4607_a3d7_739b5525f4e4.slice/crio-1c920f7d9db17842e738a0285338a2426662404a320109b3b34e0f38dcd13c19 WatchSource:0}: Error finding container 1c920f7d9db17842e738a0285338a2426662404a320109b3b34e0f38dcd13c19: Status 404 returned error can't find the container with id 1c920f7d9db17842e738a0285338a2426662404a320109b3b34e0f38dcd13c19 Jun 11 16:12:59.903776 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.903723 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-jfsm9"] Jun 11 16:12:59.908338 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:12:59.908315 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3095c8_d355_4890_b985_846e3bb2e3e4.slice/crio-67af39587cfd577ff283794d6cb9386c643f99ed3b7405ff41819185b1519602 WatchSource:0}: Error finding container 67af39587cfd577ff283794d6cb9386c643f99ed3b7405ff41819185b1519602: Status 404 returned error can't find the container with id 67af39587cfd577ff283794d6cb9386c643f99ed3b7405ff41819185b1519602 Jun 11 16:12:59.943227 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.943150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:12:59.943341 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.943262 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:12:59.943341 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.943300 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:12:59.943341 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:12:59.943331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:12:59.943480 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.943365 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls podName:4771f439-c1d3-4e27-befd-2ddafdee240e nodeName:}" failed. No retries permitted until 2026-06-11 16:13:00.943346166 +0000 UTC m=+245.308700327 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-df2r6" (UID: "4771f439-c1d3-4e27-befd-2ddafdee240e") : secret "samples-operator-tls" not found Jun 11 16:12:59.943480 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.943387 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:12:59.943480 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.943425 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert podName:209621f8-7603-4523-b0bb-d4e1f61b6913 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:00.943413527 +0000 UTC m=+245.308767677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-fqlwz" (UID: "209621f8-7603-4523-b0bb-d4e1f61b6913") : secret "networking-console-plugin-cert" not found Jun 11 16:12:59.943480 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.943442 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 16:12:59.943626 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:12:59.943491 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls podName:2b56d67a-a366-4ea7-be50-7f7f793cc700 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:00.943475367 +0000 UTC m=+245.308829520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-rfvb5" (UID: "2b56d67a-a366-4ea7-be50-7f7f793cc700") : secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:00.044137 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.044104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:00.044262 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.044243 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:13:00.044262 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.044262 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cf8c75967-2jd6s: secret "image-registry-tls" not found Jun 11 16:13:00.044339 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.044307 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls podName:201a346a-f445-40f8-8bcf-8e5cb113cc6d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:01.044293801 +0000 UTC m=+245.409647951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls") pod "image-registry-7cf8c75967-2jd6s" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d") : secret "image-registry-tls" not found Jun 11 16:13:00.751387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.750832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" event={"ID":"34dd110b-5257-42f1-9069-7c21ec59ae5d","Type":"ContainerStarted","Data":"e323e8adb9ab4763fee839179be517bf83ac783b372abb693741e817fd72cd66"} Jun 11 16:13:00.753159 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.753103 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" event={"ID":"8d690600-b47d-4607-a3d7-739b5525f4e4","Type":"ContainerStarted","Data":"1c920f7d9db17842e738a0285338a2426662404a320109b3b34e0f38dcd13c19"} Jun 11 16:13:00.756729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.756686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" event={"ID":"c9c0c5be-ef6e-4864-b60f-889eff72ad24","Type":"ContainerStarted","Data":"8875383c5fc67debc1e2a3984be2ff56fac72eb80492a4c86a6b97da5521858a"} Jun 11 16:13:00.758254 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.758214 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" event={"ID":"9a3095c8-d355-4890-b985-846e3bb2e3e4","Type":"ContainerStarted","Data":"67af39587cfd577ff283794d6cb9386c643f99ed3b7405ff41819185b1519602"} Jun 11 16:13:00.773230 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.773005 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-8nfrp" podStartSLOduration=1.772986712 podStartE2EDuration="1.772986712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:00.770823342 +0000 UTC m=+245.136177517" watchObservedRunningTime="2026-06-11 16:13:00.772986712 +0000 UTC m=+245.138340885" Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.952852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.952955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:00.952987 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.953151 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.953212 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls podName:4771f439-c1d3-4e27-befd-2ddafdee240e nodeName:}" failed. No retries permitted until 2026-06-11 16:13:02.95319346 +0000 UTC m=+247.318547615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-df2r6" (UID: "4771f439-c1d3-4e27-befd-2ddafdee240e") : secret "samples-operator-tls" not found Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.953273 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.953305 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert podName:209621f8-7603-4523-b0bb-d4e1f61b6913 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:02.953293841 +0000 UTC m=+247.318647996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-fqlwz" (UID: "209621f8-7603-4523-b0bb-d4e1f61b6913") : secret "networking-console-plugin-cert" not found Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.953355 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:00.953717 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:00.953389 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls podName:2b56d67a-a366-4ea7-be50-7f7f793cc700 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:02.953376188 +0000 UTC m=+247.318730344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-rfvb5" (UID: "2b56d67a-a366-4ea7-be50-7f7f793cc700") : secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:01.054661 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:01.054234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:01.054661 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:01.054370 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:13:01.054661 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:01.054385 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cf8c75967-2jd6s: secret "image-registry-tls" not found Jun 11 16:13:01.054661 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:01.054443 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls podName:201a346a-f445-40f8-8bcf-8e5cb113cc6d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:03.054425644 +0000 UTC m=+247.419779797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls") pod "image-registry-7cf8c75967-2jd6s" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d") : secret "image-registry-tls" not found Jun 11 16:13:01.763290 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:01.763241 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" event={"ID":"c9c0c5be-ef6e-4864-b60f-889eff72ad24","Type":"ContainerStarted","Data":"614206f50da363d95c916f0778aaa5da5845e7e7622b60a5d5e2a61ba5bfd814"} Jun 11 16:13:01.782230 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:01.782182 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-rs2xr" podStartSLOduration=1.41245027 podStartE2EDuration="2.782166716s" totalImagesPullingTime="1.369716446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:01.781451533 +0000 UTC m=+246.146805705" watchObservedRunningTime="2026-06-11 16:13:01.782166716 +0000 UTC m=+246.147520890" Jun 11 16:13:02.969433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:02.969390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:02.969511 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:02.969572 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:02.969588 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:02.969656 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls podName:4771f439-c1d3-4e27-befd-2ddafdee240e nodeName:}" failed. No retries permitted until 2026-06-11 16:13:06.969632981 +0000 UTC m=+251.334987134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-df2r6" (UID: "4771f439-c1d3-4e27-befd-2ddafdee240e") : secret "samples-operator-tls" not found Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:02.969709 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:02.969777 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert podName:209621f8-7603-4523-b0bb-d4e1f61b6913 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:06.969757933 +0000 UTC m=+251.335112086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-fqlwz" (UID: "209621f8-7603-4523-b0bb-d4e1f61b6913") : secret "networking-console-plugin-cert" not found Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:02.969709 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:02.969903 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:02.969833 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls podName:2b56d67a-a366-4ea7-be50-7f7f793cc700 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:06.969821225 +0000 UTC m=+251.335175375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-rfvb5" (UID: "2b56d67a-a366-4ea7-be50-7f7f793cc700") : secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:03.070313 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.070279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:03.070474 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:03.070441 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:13:03.070474 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:03.070462 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cf8c75967-2jd6s: secret "image-registry-tls" not found Jun 11 16:13:03.070587 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:03.070527 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls podName:201a346a-f445-40f8-8bcf-8e5cb113cc6d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:07.070506944 +0000 UTC m=+251.435861113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls") pod "image-registry-7cf8c75967-2jd6s" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d") : secret "image-registry-tls" not found Jun 11 16:13:03.771995 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.771909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" event={"ID":"79be130d-d4cc-46d4-a56c-5e216839fcc5","Type":"ContainerStarted","Data":"7f997f9f569623b7930c816822bbb38bff892ff8531a95d116ed5a76c7a456af"} Jun 11 16:13:03.773820 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.773784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-z5kgd" event={"ID":"559452f3-f51f-44c6-8997-04f31fcae598","Type":"ContainerStarted","Data":"eba10608f240769dadfb53319dbfe989f7897ce4d2dc2405d3368f414c29e57b"} Jun 11 16:13:03.775446 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.775428 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/0.log" Jun 11 16:13:03.775537 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.775463 2560 generic.go:356] "Generic (PLEG): container finished" podID="9a3095c8-d355-4890-b985-846e3bb2e3e4" containerID="45291ab71fa1e0d2128a747319a8943173b47623c989dc761983f19a40106597" exitCode=255 Jun 11 16:13:03.775600 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.775551 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" event={"ID":"9a3095c8-d355-4890-b985-846e3bb2e3e4","Type":"ContainerDied","Data":"45291ab71fa1e0d2128a747319a8943173b47623c989dc761983f19a40106597"} Jun 11 16:13:03.775729 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.775713 2560 scope.go:117] "RemoveContainer" containerID="45291ab71fa1e0d2128a747319a8943173b47623c989dc761983f19a40106597" Jun 11 16:13:03.777097 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.777069 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" event={"ID":"8d690600-b47d-4607-a3d7-739b5525f4e4","Type":"ContainerStarted","Data":"c396f53fc88938ecbebebaccf63489c1e56cdbb663421f53cfe3ce2bcec107b0"} Jun 11 16:13:03.800661 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.800618 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" podStartSLOduration=0.989271973 podStartE2EDuration="4.800604651s" totalImagesPullingTime="3.811332678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:03.799769208 +0000 UTC m=+248.165123380" watchObservedRunningTime="2026-06-11 16:13:03.800604651 +0000 UTC m=+248.165958858" Jun 11 16:13:03.831479 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.831441 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-z5kgd" podStartSLOduration=0.999389192 podStartE2EDuration="4.831424171s" totalImagesPullingTime="3.832034979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:03.829599311 +0000 UTC m=+248.194953483" watchObservedRunningTime="2026-06-11 16:13:03.831424171 +0000 UTC m=+248.196778344" Jun 11 16:13:03.904341 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:03.904060 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" podStartSLOduration=1.171624247 podStartE2EDuration="4.904037938s" totalImagesPullingTime="3.732413691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:03.861597823 +0000 UTC m=+248.226951999" watchObservedRunningTime="2026-06-11 16:13:03.904037938 +0000 UTC m=+248.269392111" Jun 11 16:13:04.781889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:04.781859 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/1.log" Jun 11 16:13:04.782351 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:04.782333 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/0.log" Jun 11 16:13:04.782397 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:04.782377 2560 generic.go:356] "Generic (PLEG): container finished" podID="9a3095c8-d355-4890-b985-846e3bb2e3e4" containerID="e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c" exitCode=255 Jun 11 16:13:04.782465 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:04.782448 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" event={"ID":"9a3095c8-d355-4890-b985-846e3bb2e3e4","Type":"ContainerDied","Data":"e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c"} Jun 11 16:13:04.782517 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:04.782496 2560 scope.go:117] "RemoveContainer" containerID="45291ab71fa1e0d2128a747319a8943173b47623c989dc761983f19a40106597" Jun 11 16:13:04.782779 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:04.782761 2560 scope.go:117] "RemoveContainer" containerID="e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c" Jun 11 16:13:04.782979 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:04.782958 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jfsm9_openshift-console-operator(9a3095c8-d355-4890-b985-846e3bb2e3e4)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" podUID="9a3095c8-d355-4890-b985-846e3bb2e3e4" Jun 11 16:13:05.787082 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:05.787054 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/1.log" Jun 11 16:13:05.787424 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:05.787352 2560 scope.go:117] "RemoveContainer" containerID="e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c" Jun 11 16:13:05.787509 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:05.787491 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jfsm9_openshift-console-operator(9a3095c8-d355-4890-b985-846e3bb2e3e4)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" podUID="9a3095c8-d355-4890-b985-846e3bb2e3e4" Jun 11 16:13:07.004364 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.004320 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.004375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.004403 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.004489 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.004498 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.004507 2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.004567 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert podName:209621f8-7603-4523-b0bb-d4e1f61b6913 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:15.004544141 +0000 UTC m=+259.369898295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-fqlwz" (UID: "209621f8-7603-4523-b0bb-d4e1f61b6913") : secret "networking-console-plugin-cert" not found Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.004590 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls podName:2b56d67a-a366-4ea7-be50-7f7f793cc700 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:15.004579372 +0000 UTC m=+259.369933522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-rfvb5" (UID: "2b56d67a-a366-4ea7-be50-7f7f793cc700") : secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:07.004786 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.004607 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls podName:4771f439-c1d3-4e27-befd-2ddafdee240e nodeName:}" failed. No retries permitted until 2026-06-11 16:13:15.004598422 +0000 UTC m=+259.369952578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-df2r6" (UID: "4771f439-c1d3-4e27-befd-2ddafdee240e") : secret "samples-operator-tls" not found Jun 11 16:13:07.105612 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.105580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:07.105742 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.105718 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:13:07.105742 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.105734 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cf8c75967-2jd6s: secret "image-registry-tls" not found Jun 11 16:13:07.105816 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.105782 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls podName:201a346a-f445-40f8-8bcf-8e5cb113cc6d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:15.105765241 +0000 UTC m=+259.471119393 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls") pod "image-registry-7cf8c75967-2jd6s" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d") : secret "image-registry-tls" not found Jun 11 16:13:07.383720 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.383648 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-48xhk"] Jun 11 16:13:07.386390 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.386374 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.390097 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.390077 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 11 16:13:07.390205 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.390137 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 11 16:13:07.390672 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.390649 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 11 16:13:07.391318 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.391300 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5j6zf\"" Jun 11 16:13:07.391401 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.391302 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 11 16:13:07.401984 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.401957 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-48xhk"] Jun 11 16:13:07.508963 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.508923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/68188b2d-5507-4811-806a-150d200c329a-signing-cabundle\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.509078 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.508984 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn898\" (UniqueName: \"kubernetes.io/projected/68188b2d-5507-4811-806a-150d200c329a-kube-api-access-vn898\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.509078 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.509030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/68188b2d-5507-4811-806a-150d200c329a-signing-key\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.610149 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.610123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/68188b2d-5507-4811-806a-150d200c329a-signing-cabundle\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.610244 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.610160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn898\" (UniqueName: \"kubernetes.io/projected/68188b2d-5507-4811-806a-150d200c329a-kube-api-access-vn898\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.610244 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.610221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/68188b2d-5507-4811-806a-150d200c329a-signing-key\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.610863 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.610833 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/68188b2d-5507-4811-806a-150d200c329a-signing-cabundle\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.612652 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.612635 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/68188b2d-5507-4811-806a-150d200c329a-signing-key\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.619199 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.619182 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn898\" (UniqueName: \"kubernetes.io/projected/68188b2d-5507-4811-806a-150d200c329a-kube-api-access-vn898\") pod \"service-ca-7d74d578b8-48xhk\" (UID: \"68188b2d-5507-4811-806a-150d200c329a\") " pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.695179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.695128 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" Jun 11 16:13:07.832588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.832560 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-48xhk"] Jun 11 16:13:07.835641 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:07.835619 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68188b2d_5507_4811_806a_150d200c329a.slice/crio-c8f749c2b378a6e45ddc9127c85e4d9caa3d5f62bfdeb3c810a920f3dad6bac4 WatchSource:0}: Error finding container c8f749c2b378a6e45ddc9127c85e4d9caa3d5f62bfdeb3c810a920f3dad6bac4: Status 404 returned error can't find the container with id c8f749c2b378a6e45ddc9127c85e4d9caa3d5f62bfdeb3c810a920f3dad6bac4 Jun 11 16:13:07.912987 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:07.912963 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:13:07.913096 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.913085 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:13:07.913160 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:07.913147 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs podName:72260f98-ae30-4e5d-ac3f-989e32672507 nodeName:}" failed. No retries permitted until 2026-06-11 16:15:09.913126828 +0000 UTC m=+374.278480980 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs") pod "network-metrics-daemon-77sz2" (UID: "72260f98-ae30-4e5d-ac3f-989e32672507") : secret "metrics-daemon-secret" not found Jun 11 16:13:08.226839 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:08.226811 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-275rc_ad8f72cc-af12-44c9-9715-c07563601aee/dns-node-resolver/0.log" Jun 11 16:13:08.800820 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:08.800784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" event={"ID":"68188b2d-5507-4811-806a-150d200c329a","Type":"ContainerStarted","Data":"5b5294991bcf060f80549aa02ab14393615c5014971151e7735f783c7a3fe98e"} Jun 11 16:13:08.801001 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:08.800842 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" event={"ID":"68188b2d-5507-4811-806a-150d200c329a","Type":"ContainerStarted","Data":"c8f749c2b378a6e45ddc9127c85e4d9caa3d5f62bfdeb3c810a920f3dad6bac4"} Jun 11 16:13:08.838807 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:08.838760 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-48xhk" podStartSLOduration=1.838745915 podStartE2EDuration="1.838745915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:08.837624355 +0000 UTC m=+253.202978526" watchObservedRunningTime="2026-06-11 16:13:08.838745915 +0000 UTC m=+253.204100114" Jun 11 16:13:09.629110 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:09.629081 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pcqw6_03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf/node-ca/0.log" Jun 11 16:13:09.754962 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:09.754915 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:13:09.755112 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:09.754981 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:13:09.755268 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:09.755256 2560 scope.go:117] "RemoveContainer" containerID="e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c" Jun 11 16:13:09.755415 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:09.755397 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-jfsm9_openshift-console-operator(9a3095c8-d355-4890-b985-846e3bb2e3e4)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" podUID="9a3095c8-d355-4890-b985-846e3bb2e3e4" Jun 11 16:13:10.041540 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:10.041512 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7bmts_73dcc6de-fa1c-4145-a113-653c0863671e/serve-healthcheck-canary/0.log" Jun 11 16:13:11.028011 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:11.027985 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-xv5fk_79be130d-d4cc-46d4-a56c-5e216839fcc5/kube-storage-version-migrator-operator/0.log" Jun 11 16:13:15.066950 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.066848 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:15.066950 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.066915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:15.067444 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.066965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:13:15.067444 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:15.067008 2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 11 16:13:15.067444 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:15.067071 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert podName:209621f8-7603-4523-b0bb-d4e1f61b6913 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:31.067056321 +0000 UTC m=+275.432410471 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-fqlwz" (UID: "209621f8-7603-4523-b0bb-d4e1f61b6913") : secret "networking-console-plugin-cert" not found Jun 11 16:13:15.067444 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:15.067150 2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:15.067444 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:15.067202 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls podName:2b56d67a-a366-4ea7-be50-7f7f793cc700 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:31.067185631 +0000 UTC m=+275.432539780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-74bbf69bbb-rfvb5" (UID: "2b56d67a-a366-4ea7-be50-7f7f793cc700") : secret "cluster-monitoring-operator-tls" not found Jun 11 16:13:15.069518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.069497 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4771f439-c1d3-4e27-befd-2ddafdee240e-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-df2r6\" (UID: \"4771f439-c1d3-4e27-befd-2ddafdee240e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:13:15.167482 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.167455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:15.169797 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.169778 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"image-registry-7cf8c75967-2jd6s\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:15.197805 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.197780 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" Jun 11 16:13:15.204740 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.204723 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:15.340784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.340709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6"] Jun 11 16:13:15.361827 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.361801 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cf8c75967-2jd6s"] Jun 11 16:13:15.364175 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:15.364149 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201a346a_f445_40f8_8bcf_8e5cb113cc6d.slice/crio-e1d9ec7b6c1abaed4ed5ae6b12939fc7c0924133f6656e5a3fdcbd0e67029b8b WatchSource:0}: Error finding container e1d9ec7b6c1abaed4ed5ae6b12939fc7c0924133f6656e5a3fdcbd0e67029b8b: Status 404 returned error can't find the container with id e1d9ec7b6c1abaed4ed5ae6b12939fc7c0924133f6656e5a3fdcbd0e67029b8b Jun 11 16:13:15.827790 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.827758 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" event={"ID":"201a346a-f445-40f8-8bcf-8e5cb113cc6d","Type":"ContainerStarted","Data":"c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11"} Jun 11 16:13:15.828053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.827813 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" event={"ID":"201a346a-f445-40f8-8bcf-8e5cb113cc6d","Type":"ContainerStarted","Data":"e1d9ec7b6c1abaed4ed5ae6b12939fc7c0924133f6656e5a3fdcbd0e67029b8b"} Jun 11 16:13:15.828053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.827856 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:15.829033 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.829000 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" event={"ID":"4771f439-c1d3-4e27-befd-2ddafdee240e","Type":"ContainerStarted","Data":"401dac5ad7ff3da6be18e1f92d838304d16af31480055f1a8b4940ebe5f2e44e"} Jun 11 16:13:15.865674 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:15.865633 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" podStartSLOduration=16.865622773 podStartE2EDuration="16.865622773s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:15.864066099 +0000 UTC m=+260.229420272" watchObservedRunningTime="2026-06-11 16:13:15.865622773 +0000 UTC m=+260.230976975" Jun 11 16:13:17.837104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:17.837074 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" event={"ID":"4771f439-c1d3-4e27-befd-2ddafdee240e","Type":"ContainerStarted","Data":"9624267d592d8cf32629e9db1eb2b74be02ef173fb85025d0ebddf45ed818ec6"} Jun 11 16:13:17.837434 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:17.837128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" event={"ID":"4771f439-c1d3-4e27-befd-2ddafdee240e","Type":"ContainerStarted","Data":"aff3937196ac6d4c270e3e487f79adde0edb77551e6f2f0d04dbd500e374af78"} Jun 11 16:13:17.872037 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:17.871992 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-df2r6" podStartSLOduration=16.823345423 podStartE2EDuration="18.871978562s" totalImagesPullingTime="2.048633139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:17.871642541 +0000 UTC m=+262.236996723" watchObservedRunningTime="2026-06-11 16:13:17.871978562 +0000 UTC m=+262.237332791" Jun 11 16:13:20.061239 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.061201 2560 scope.go:117] "RemoveContainer" containerID="e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c" Jun 11 16:13:20.848907 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.848879 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:13:20.849343 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.849327 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/1.log" Jun 11 16:13:20.849420 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.849359 2560 generic.go:356] "Generic (PLEG): container finished" podID="9a3095c8-d355-4890-b985-846e3bb2e3e4" containerID="4cba34f4fe319358ce094ded3d0e59ba98777c9b25461b68ad6dec9c0113ed4a" exitCode=255 Jun 11 16:13:20.849420 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.849411 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" event={"ID":"9a3095c8-d355-4890-b985-846e3bb2e3e4","Type":"ContainerDied","Data":"4cba34f4fe319358ce094ded3d0e59ba98777c9b25461b68ad6dec9c0113ed4a"} Jun 11 16:13:20.849493 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.849465 2560 scope.go:117] "RemoveContainer" containerID="e992e5ef6283798733f1cda74e0b7129a5d99c12b25287846a553b6f071cf87c" Jun 11 16:13:20.849702 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:20.849688 2560 scope.go:117] "RemoveContainer" containerID="4cba34f4fe319358ce094ded3d0e59ba98777c9b25461b68ad6dec9c0113ed4a" Jun 11 16:13:20.849858 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:20.849838 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-jfsm9_openshift-console-operator(9a3095c8-d355-4890-b985-846e3bb2e3e4)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" podUID="9a3095c8-d355-4890-b985-846e3bb2e3e4" Jun 11 16:13:21.853668 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:21.853635 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:13:29.754657 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:29.754608 2560 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:13:29.755122 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:29.754699 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:13:29.755122 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:29.755017 2560 scope.go:117] "RemoveContainer" containerID="4cba34f4fe319358ce094ded3d0e59ba98777c9b25461b68ad6dec9c0113ed4a" Jun 11 16:13:29.755194 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:29.755166 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-jfsm9_openshift-console-operator(9a3095c8-d355-4890-b985-846e3bb2e3e4)\"" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" podUID="9a3095c8-d355-4890-b985-846e3bb2e3e4" Jun 11 16:13:30.225381 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.225304 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4kmvj"] Jun 11 16:13:30.231162 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.231140 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.244568 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.244547 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 16:13:30.244683 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.244579 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 16:13:30.246677 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.246662 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sksvz\"" Jun 11 16:13:30.252644 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.252623 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4kmvj"] Jun 11 16:13:30.272377 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.272357 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cf8c75967-2jd6s"] Jun 11 16:13:30.276755 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.276727 2560 patch_prober.go:28] interesting pod/image-registry-7cf8c75967-2jd6s container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 16:13:30.276846 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.276789 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:13:30.283609 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.283584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83da3d-65b6-496c-a9c5-c2efd392748e-data-volume\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.283714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.283613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83da3d-65b6-496c-a9c5-c2efd392748e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.283714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.283636 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49pv\" (UniqueName: \"kubernetes.io/projected/7d83da3d-65b6-496c-a9c5-c2efd392748e-kube-api-access-c49pv\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.283823 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.283722 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83da3d-65b6-496c-a9c5-c2efd392748e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.283823 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.283768 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83da3d-65b6-496c-a9c5-c2efd392748e-crio-socket\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.361591 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.361562 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-658f6f668-s6lng"] Jun 11 16:13:30.365522 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.365506 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.384626 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83da3d-65b6-496c-a9c5-c2efd392748e-data-volume\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.384714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83da3d-65b6-496c-a9c5-c2efd392748e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.384714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49pv\" (UniqueName: \"kubernetes.io/projected/7d83da3d-65b6-496c-a9c5-c2efd392748e-kube-api-access-c49pv\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.384714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0101db04-d384-4db3-a0b8-d17758cf3c00-trusted-ca\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.384714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-bound-sa-token\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.384869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn8s2\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-kube-api-access-dn8s2\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.384869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384755 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83da3d-65b6-496c-a9c5-c2efd392748e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.384869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0101db04-d384-4db3-a0b8-d17758cf3c00-installation-pull-secrets\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.384869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384867 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0101db04-d384-4db3-a0b8-d17758cf3c00-registry-certificates\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.385057 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.384910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83da3d-65b6-496c-a9c5-c2efd392748e-crio-socket\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.385057 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.385004 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7d83da3d-65b6-496c-a9c5-c2efd392748e-crio-socket\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.385057 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.385039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0101db04-d384-4db3-a0b8-d17758cf3c00-image-registry-private-configuration\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.385201 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.385071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-registry-tls\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.385201 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.385077 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7d83da3d-65b6-496c-a9c5-c2efd392748e-data-volume\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.385201 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.385097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0101db04-d384-4db3-a0b8-d17758cf3c00-ca-trust-extracted\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.385309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.385275 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7d83da3d-65b6-496c-a9c5-c2efd392748e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.387272 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.387251 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7d83da3d-65b6-496c-a9c5-c2efd392748e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.393707 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.393686 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658f6f668-s6lng"] Jun 11 16:13:30.426398 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.426368 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49pv\" (UniqueName: \"kubernetes.io/projected/7d83da3d-65b6-496c-a9c5-c2efd392748e-kube-api-access-c49pv\") pod \"insights-runtime-extractor-4kmvj\" (UID: \"7d83da3d-65b6-496c-a9c5-c2efd392748e\") " pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.486133 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486063 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0101db04-d384-4db3-a0b8-d17758cf3c00-trusted-ca\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486133 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-bound-sa-token\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486313 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486158 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn8s2\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-kube-api-access-dn8s2\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486313 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486185 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0101db04-d384-4db3-a0b8-d17758cf3c00-installation-pull-secrets\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486313 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0101db04-d384-4db3-a0b8-d17758cf3c00-registry-certificates\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486394 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0101db04-d384-4db3-a0b8-d17758cf3c00-image-registry-private-configuration\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-registry-tls\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.486570 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.486464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0101db04-d384-4db3-a0b8-d17758cf3c00-ca-trust-extracted\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.487083 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.487060 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0101db04-d384-4db3-a0b8-d17758cf3c00-ca-trust-extracted\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.487209 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.487092 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0101db04-d384-4db3-a0b8-d17758cf3c00-registry-certificates\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.487656 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.487634 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0101db04-d384-4db3-a0b8-d17758cf3c00-trusted-ca\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.488871 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.488849 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0101db04-d384-4db3-a0b8-d17758cf3c00-installation-pull-secrets\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.489488 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.489470 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-registry-tls\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.489580 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.489558 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0101db04-d384-4db3-a0b8-d17758cf3c00-image-registry-private-configuration\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.502912 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.502886 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-bound-sa-token\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.505179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.505157 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn8s2\" (UniqueName: \"kubernetes.io/projected/0101db04-d384-4db3-a0b8-d17758cf3c00-kube-api-access-dn8s2\") pod \"image-registry-658f6f668-s6lng\" (UID: \"0101db04-d384-4db3-a0b8-d17758cf3c00\") " pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.539499 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.539477 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4kmvj" Jun 11 16:13:30.674301 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.674276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:30.677845 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.677824 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4kmvj"] Jun 11 16:13:30.680257 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:30.680223 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d83da3d_65b6_496c_a9c5_c2efd392748e.slice/crio-8d13b86cbb5a2d7cea8f6dcefa54c3ff8f6901aba702c08ac89c03d92bb522e2 WatchSource:0}: Error finding container 8d13b86cbb5a2d7cea8f6dcefa54c3ff8f6901aba702c08ac89c03d92bb522e2: Status 404 returned error can't find the container with id 8d13b86cbb5a2d7cea8f6dcefa54c3ff8f6901aba702c08ac89c03d92bb522e2 Jun 11 16:13:30.816079 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.816053 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-658f6f668-s6lng"] Jun 11 16:13:30.818209 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:30.818186 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0101db04_d384_4db3_a0b8_d17758cf3c00.slice/crio-e3545f818b451cb2ea9c40b03594a55d0afb359234c52a0530b278ed8bb344ae WatchSource:0}: Error finding container e3545f818b451cb2ea9c40b03594a55d0afb359234c52a0530b278ed8bb344ae: Status 404 returned error can't find the container with id e3545f818b451cb2ea9c40b03594a55d0afb359234c52a0530b278ed8bb344ae Jun 11 16:13:30.884633 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.884599 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kmvj" event={"ID":"7d83da3d-65b6-496c-a9c5-c2efd392748e","Type":"ContainerStarted","Data":"fd8354c20ce756c23d18b53c42a5d4272ca91ed046f677c172cc848782ccea78"} Jun 11 16:13:30.884739 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.884672 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kmvj" event={"ID":"7d83da3d-65b6-496c-a9c5-c2efd392748e","Type":"ContainerStarted","Data":"8d13b86cbb5a2d7cea8f6dcefa54c3ff8f6901aba702c08ac89c03d92bb522e2"} Jun 11 16:13:30.886087 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:30.886059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658f6f668-s6lng" event={"ID":"0101db04-d384-4db3-a0b8-d17758cf3c00","Type":"ContainerStarted","Data":"e3545f818b451cb2ea9c40b03594a55d0afb359234c52a0530b278ed8bb344ae"} Jun 11 16:13:31.091311 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.091230 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:31.091470 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.091330 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:31.093629 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.093602 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2b56d67a-a366-4ea7-be50-7f7f793cc700-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-74bbf69bbb-rfvb5\" (UID: \"2b56d67a-a366-4ea7-be50-7f7f793cc700\") " pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:31.093725 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.093674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/209621f8-7603-4523-b0bb-d4e1f61b6913-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-fqlwz\" (UID: \"209621f8-7603-4523-b0bb-d4e1f61b6913\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:31.297667 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.297639 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" Jun 11 16:13:31.318658 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.318636 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" Jun 11 16:13:31.444690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.444616 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz"] Jun 11 16:13:31.477261 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.477237 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5"] Jun 11 16:13:31.533672 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:31.533645 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209621f8_7603_4523_b0bb_d4e1f61b6913.slice/crio-8e9967f8c716f12df13b2c8cd8f63a50f550a1a2fc7a7703953bd18fb6130a2e WatchSource:0}: Error finding container 8e9967f8c716f12df13b2c8cd8f63a50f550a1a2fc7a7703953bd18fb6130a2e: Status 404 returned error can't find the container with id 8e9967f8c716f12df13b2c8cd8f63a50f550a1a2fc7a7703953bd18fb6130a2e Jun 11 16:13:31.534373 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:31.534353 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b56d67a_a366_4ea7_be50_7f7f793cc700.slice/crio-f0aed87c81d549c056e7b917cbc8aeceeefd11103d29bf60debe6642a8eec193 WatchSource:0}: Error finding container f0aed87c81d549c056e7b917cbc8aeceeefd11103d29bf60debe6642a8eec193: Status 404 returned error can't find the container with id f0aed87c81d549c056e7b917cbc8aeceeefd11103d29bf60debe6642a8eec193 Jun 11 16:13:31.889621 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.889595 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" event={"ID":"2b56d67a-a366-4ea7-be50-7f7f793cc700","Type":"ContainerStarted","Data":"f0aed87c81d549c056e7b917cbc8aeceeefd11103d29bf60debe6642a8eec193"} Jun 11 16:13:31.890968 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.890922 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-658f6f668-s6lng" event={"ID":"0101db04-d384-4db3-a0b8-d17758cf3c00","Type":"ContainerStarted","Data":"52cfabfebaa3542dce39b4c4c81fc879752084bf33373b8211ab8e510d84c5b0"} Jun 11 16:13:31.891870 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.891849 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" event={"ID":"209621f8-7603-4523-b0bb-d4e1f61b6913","Type":"ContainerStarted","Data":"8e9967f8c716f12df13b2c8cd8f63a50f550a1a2fc7a7703953bd18fb6130a2e"} Jun 11 16:13:31.899864 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.899848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:31.921133 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:31.920988 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-658f6f668-s6lng" podStartSLOduration=1.920972598 podStartE2EDuration="1.920972598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:31.920662493 +0000 UTC m=+276.286016666" watchObservedRunningTime="2026-06-11 16:13:31.920972598 +0000 UTC m=+276.286326772" Jun 11 16:13:32.898667 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:32.898618 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kmvj" event={"ID":"7d83da3d-65b6-496c-a9c5-c2efd392748e","Type":"ContainerStarted","Data":"ee382118b5980d12e2eaaccf299bb663ef4f22bb3a3a5d15184d07a68e2c28ed"} Jun 11 16:13:34.907484 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:34.907442 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4kmvj" event={"ID":"7d83da3d-65b6-496c-a9c5-c2efd392748e","Type":"ContainerStarted","Data":"b946099d63378a19383ea97cb2c7ce7158cce8165612a0ac48a519f3e4afe392"} Jun 11 16:13:34.908848 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:34.908815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" event={"ID":"2b56d67a-a366-4ea7-be50-7f7f793cc700","Type":"ContainerStarted","Data":"048a300a6c268894cb54a987124ff7c44a0cf342c7b39dc6271797d45e6f925c"} Jun 11 16:13:34.910119 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:34.910087 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" event={"ID":"209621f8-7603-4523-b0bb-d4e1f61b6913","Type":"ContainerStarted","Data":"09086455bbd842286b4eca41435384770552ba033ff35cdb3c8cb095a5c6bcff"} Jun 11 16:13:34.936886 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:34.936841 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4kmvj" podStartSLOduration=1.430274708 podStartE2EDuration="4.936829217s" totalImagesPullingTime="3.506554509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:13:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:34.935823291 +0000 UTC m=+279.301177463" watchObservedRunningTime="2026-06-11 16:13:34.936829217 +0000 UTC m=+279.302183389" Jun 11 16:13:34.965409 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:34.965365 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-fqlwz" podStartSLOduration=33.124910078 podStartE2EDuration="35.965350688s" totalImagesPullingTime="2.84044061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:34.963741147 +0000 UTC m=+279.329095319" watchObservedRunningTime="2026-06-11 16:13:34.965350688 +0000 UTC m=+279.330704903" Jun 11 16:13:34.981889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:34.981837 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-74bbf69bbb-rfvb5" podStartSLOduration=33.144881929 podStartE2EDuration="35.981821233s" totalImagesPullingTime="2.836939304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:34.98085321 +0000 UTC m=+279.346207382" watchObservedRunningTime="2026-06-11 16:13:34.981821233 +0000 UTC m=+279.347175449" Jun 11 16:13:40.062918 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.062870 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:13:40.065507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.065476 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/09604985-6b11-49bc-98d6-1331719d4f16-metrics-tls\") pod \"dns-default-z2szh\" (UID: \"09604985-6b11-49bc-98d6-1331719d4f16\") " pod="openshift-dns/dns-default-z2szh" Jun 11 16:13:40.264812 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.264780 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pbxmp\"" Jun 11 16:13:40.272291 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.272271 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-z2szh" Jun 11 16:13:40.277505 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.277481 2560 patch_prober.go:28] interesting pod/image-registry-7cf8c75967-2jd6s container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 16:13:40.277618 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.277522 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:13:40.411630 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.411599 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-z2szh"] Jun 11 16:13:40.414765 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:40.414734 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09604985_6b11_49bc_98d6_1331719d4f16.slice/crio-322e8ffad594c88a2a7db8af48ab785556a56ae5ba90124954f0d073309701b1 WatchSource:0}: Error finding container 322e8ffad594c88a2a7db8af48ab785556a56ae5ba90124954f0d073309701b1: Status 404 returned error can't find the container with id 322e8ffad594c88a2a7db8af48ab785556a56ae5ba90124954f0d073309701b1 Jun 11 16:13:40.932131 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:40.932094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z2szh" event={"ID":"09604985-6b11-49bc-98d6-1331719d4f16","Type":"ContainerStarted","Data":"322e8ffad594c88a2a7db8af48ab785556a56ae5ba90124954f0d073309701b1"} Jun 11 16:13:41.936739 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:41.936712 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z2szh" event={"ID":"09604985-6b11-49bc-98d6-1331719d4f16","Type":"ContainerStarted","Data":"c3f76a824315db6cf430abe3e1c775fc75940b566b534aa9f3335c6a7de40f90"} Jun 11 16:13:41.937113 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:41.936761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-z2szh" event={"ID":"09604985-6b11-49bc-98d6-1331719d4f16","Type":"ContainerStarted","Data":"7a9de3eb9b0e9a0e524bc8855e95a1766352454c9a890110efd0a5f1309fdf46"} Jun 11 16:13:41.937113 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:41.936833 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-z2szh" Jun 11 16:13:41.955966 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:41.955910 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-z2szh" podStartSLOduration=250.615486801 podStartE2EDuration="4m11.95589564s" totalImagesPullingTime="1.340408839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:41.95497169 +0000 UTC m=+286.320325863" watchObservedRunningTime="2026-06-11 16:13:41.95589564 +0000 UTC m=+286.321249824" Jun 11 16:13:43.060503 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.060466 2560 scope.go:117] "RemoveContainer" containerID="4cba34f4fe319358ce094ded3d0e59ba98777c9b25461b68ad6dec9c0113ed4a" Jun 11 16:13:43.571565 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.571531 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5mn7j"] Jun 11 16:13:43.574272 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.574192 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.577270 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.577228 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 16:13:43.577783 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.577760 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xfzx2\"" Jun 11 16:13:43.577877 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.577601 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 16:13:43.577943 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.577898 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 16:13:43.578890 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.578718 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 16:13:43.593338 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593245 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-textfile\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593338 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-sys\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593338 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-wtmp\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593543 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593383 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-tls\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593543 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593543 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593514 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzz2p\" (UniqueName: \"kubernetes.io/projected/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-kube-api-access-zzz2p\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-root\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-metrics-client-ca\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.593782 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.593684 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.695134 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695092 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-textfile\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.695378 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695356 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-sys\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.695501 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695473 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-sys\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.695580 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695542 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-textfile\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.695695 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-wtmp\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.695833 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695819 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-tls\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.696101 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.695691 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-wtmp\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.696210 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:43.696031 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 16:13:43.696210 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:43.696188 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-tls podName:3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f nodeName:}" failed. No retries permitted until 2026-06-11 16:13:44.196164382 +0000 UTC m=+288.561518546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-tls") pod "node-exporter-5mn7j" (UID: "3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f") : secret "node-exporter-tls" not found Jun 11 16:13:43.696907 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.696864 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.697055 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.696083 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.697183 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.697168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzz2p\" (UniqueName: \"kubernetes.io/projected/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-kube-api-access-zzz2p\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.697351 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.697338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-root\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.697533 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.697513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-metrics-client-ca\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.697736 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.697473 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-root\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.698005 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.697692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.698181 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.698157 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-metrics-client-ca\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.700333 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.700310 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.711583 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.711556 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzz2p\" (UniqueName: \"kubernetes.io/projected/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-kube-api-access-zzz2p\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:43.947677 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.947596 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:13:43.947828 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.947688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" event={"ID":"9a3095c8-d355-4890-b985-846e3bb2e3e4","Type":"ContainerStarted","Data":"6bc3e017311b110ebe145b31516f771ad53b1e0ff31beddb25f55c423804833f"} Jun 11 16:13:43.947995 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.947976 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:13:43.953415 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.953394 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" Jun 11 16:13:43.974813 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:43.974773 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-jfsm9" podStartSLOduration=41.358519594 podStartE2EDuration="44.974756607s" totalImagesPullingTime="3.616237013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:43.972591581 +0000 UTC m=+288.337945753" watchObservedRunningTime="2026-06-11 16:13:43.974756607 +0000 UTC m=+288.340110783" Jun 11 16:13:44.202730 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.202651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-tls\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:44.204970 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.204942 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f-node-exporter-tls\") pod \"node-exporter-5mn7j\" (UID: \"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f\") " pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:44.487085 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.487019 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5mn7j" Jun 11 16:13:44.497429 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:44.497402 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6bf72c_b5e1_47e1_83b6_bcea3bafdb9f.slice/crio-0b9be7365dad6313c5d3118b5c7370b82a1347d1a983dcf32867f41da2442b04 WatchSource:0}: Error finding container 0b9be7365dad6313c5d3118b5c7370b82a1347d1a983dcf32867f41da2442b04: Status 404 returned error can't find the container with id 0b9be7365dad6313c5d3118b5c7370b82a1347d1a983dcf32867f41da2442b04 Jun 11 16:13:44.712076 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.712038 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:13:44.714743 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.714719 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.719464 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.719447 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 11 16:13:44.719629 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.719448 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 11 16:13:44.719714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.719632 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 11 16:13:44.719714 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.719445 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 11 16:13:44.719817 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.719713 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lb2mt\"" Jun 11 16:13:44.719817 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.719741 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 11 16:13:44.720104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.720008 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 11 16:13:44.720104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.720049 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 11 16:13:44.720285 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.720112 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 11 16:13:44.720285 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.720155 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 11 16:13:44.758262 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.758239 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:13:44.808031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808059 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808088 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808110 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-config-volume\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808156 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-config-out\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808218 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-web-config\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808468 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808284 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pq89\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-kube-api-access-5pq89\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808468 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808336 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.808468 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.808384 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.908802 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.908761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.908979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.908835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.908979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.908861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.908979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.908885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-config-volume\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.908979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.908923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.908980 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-config-out\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909032 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909069 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-web-config\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909096 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pq89\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-kube-api-access-5pq89\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909212 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909171 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909547 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.909703 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.909681 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.910921 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.910564 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.911559 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.911508 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.912477 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.912450 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.912981 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.912918 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-web-config\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.914173 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.914127 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.914309 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.914287 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.914505 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.914483 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.914578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.914524 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.914889 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.914863 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-config-volume\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.915662 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.915637 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.915922 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.915901 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-config-out\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.919873 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.919733 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pq89\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-kube-api-access-5pq89\") pod \"alertmanager-main-0\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:44.953663 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:44.953627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mn7j" event={"ID":"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f","Type":"ContainerStarted","Data":"0b9be7365dad6313c5d3118b5c7370b82a1347d1a983dcf32867f41da2442b04"} Jun 11 16:13:45.026557 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:45.026462 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:13:45.194814 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:45.194777 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:13:45.288112 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:45.288044 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd574abce_3120_4d7b_bab6_1315f2971c53.slice/crio-a518f4d5e69c1df50679bf24afc6084e44dcc79035d9ba4d61f33db4eedf7a2c WatchSource:0}: Error finding container a518f4d5e69c1df50679bf24afc6084e44dcc79035d9ba4d61f33db4eedf7a2c: Status 404 returned error can't find the container with id a518f4d5e69c1df50679bf24afc6084e44dcc79035d9ba4d61f33db4eedf7a2c Jun 11 16:13:45.958621 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:45.958584 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"a518f4d5e69c1df50679bf24afc6084e44dcc79035d9ba4d61f33db4eedf7a2c"} Jun 11 16:13:45.960136 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:45.960102 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f" containerID="65e24bcdcafa246922a93d372240116caf34f97708d0e24b32a517f09e3b1cc0" exitCode=0 Jun 11 16:13:45.960268 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:45.960189 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mn7j" event={"ID":"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f","Type":"ContainerDied","Data":"65e24bcdcafa246922a93d372240116caf34f97708d0e24b32a517f09e3b1cc0"} Jun 11 16:13:46.967647 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:46.967621 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mn7j" event={"ID":"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f","Type":"ContainerStarted","Data":"da1d3cf54c39c1c1d16c4fd9556ae9429a6224f8d75a3948b39a68e59b51412f"} Jun 11 16:13:46.967995 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:46.967686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mn7j" event={"ID":"3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f","Type":"ContainerStarted","Data":"8e0609745c02c07969710b45dce73ec6bd29ac50500b82db73b77985d700ba20"} Jun 11 16:13:46.969232 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:46.969209 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"dec5b2ba9c3671af04cc8d2795656f0fad2a13c6192dcc353f0fb29c6591bbe9"} Jun 11 16:13:46.988649 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:46.988601 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5mn7j" podStartSLOduration=3.106265588 podStartE2EDuration="3.988584336s" totalImagesPullingTime="810.967195ms" totalInitContainerRuntime="71.351553ms" isStatefulPod=true podCreationTimestamp="2026-06-11 16:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:46.988523645 +0000 UTC m=+291.353877820" watchObservedRunningTime="2026-06-11 16:13:46.988584336 +0000 UTC m=+291.353938510" Jun 11 16:13:47.975129 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:47.975092 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="dec5b2ba9c3671af04cc8d2795656f0fad2a13c6192dcc353f0fb29c6591bbe9" exitCode=0 Jun 11 16:13:47.975553 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:47.975171 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"dec5b2ba9c3671af04cc8d2795656f0fad2a13c6192dcc353f0fb29c6591bbe9"} Jun 11 16:13:48.942540 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.942503 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9bbf5687c-5cfgj"] Jun 11 16:13:48.947037 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.947018 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:48.949555 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.949536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 11 16:13:48.951127 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.951108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 11 16:13:48.953053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.953037 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 11 16:13:48.953230 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.953211 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 11 16:13:48.953966 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.953949 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fk8ww\"" Jun 11 16:13:48.954129 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.954117 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 11 16:13:48.982596 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.982573 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"e1235c9397acb24b7bbfe874cb9e27ee3c90d05e870b1e9657470016ffb291c6"} Jun 11 16:13:48.982949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.982640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"03cbdc26df9d47cf29407b20f29470187f184ab2ffa71af54e86c9513d4c1d23"} Jun 11 16:13:48.982949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.982667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"13f9f096535b0a4a4b232ff048897461bf389bddd60d713b20e84e31915cc004"} Jun 11 16:13:48.982949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.982697 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"e8525db0484edd6820883d10d1c3e09a26415380a72f71c3ecb39c37c55917fc"} Jun 11 16:13:48.986047 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:48.986028 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 11 16:13:49.002850 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.002829 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9bbf5687c-5cfgj"] Jun 11 16:13:49.043786 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.043763 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgsqs\" (UniqueName: \"kubernetes.io/projected/8741f554-4e17-4c65-84e7-ac9221141bbc-kube-api-access-xgsqs\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.043885 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.043793 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-metrics-client-ca\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.043885 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.043815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-telemeter-client-tls\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.043982 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.043891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.043982 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.043944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-serving-certs-ca-bundle\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.044052 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.043986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-secret-telemeter-client\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.044052 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.044002 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.044110 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.044063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-federate-client-tls\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.144979 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.144955 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-federate-client-tls\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145082 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.144998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgsqs\" (UniqueName: \"kubernetes.io/projected/8741f554-4e17-4c65-84e7-ac9221141bbc-kube-api-access-xgsqs\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145082 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145019 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-metrics-client-ca\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145082 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-telemeter-client-tls\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145291 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145291 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-serving-certs-ca-bundle\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145291 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-secret-telemeter-client\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.145445 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145345 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.146232 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.145770 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-metrics-client-ca\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.146313 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.146122 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-serving-certs-ca-bundle\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.146375 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.146354 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8741f554-4e17-4c65-84e7-ac9221141bbc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.147788 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.147766 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.147854 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.147773 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-federate-client-tls\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.148051 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.148034 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-telemeter-client-tls\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.148098 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.148071 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8741f554-4e17-4c65-84e7-ac9221141bbc-secret-telemeter-client\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.157763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.157744 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgsqs\" (UniqueName: \"kubernetes.io/projected/8741f554-4e17-4c65-84e7-ac9221141bbc-kube-api-access-xgsqs\") pod \"telemeter-client-9bbf5687c-5cfgj\" (UID: \"8741f554-4e17-4c65-84e7-ac9221141bbc\") " pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.282183 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.282161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" Jun 11 16:13:49.447848 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.447818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9bbf5687c-5cfgj"] Jun 11 16:13:49.450352 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:49.450323 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8741f554_4e17_4c65_84e7_ac9221141bbc.slice/crio-f92990199e3ce9e7b649e3f10732f8385cb8366fe7ef2592a26f154a044e4de5 WatchSource:0}: Error finding container f92990199e3ce9e7b649e3f10732f8385cb8366fe7ef2592a26f154a044e4de5: Status 404 returned error can't find the container with id f92990199e3ce9e7b649e3f10732f8385cb8366fe7ef2592a26f154a044e4de5 Jun 11 16:13:49.895691 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.895658 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:13:49.899297 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.899281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.902543 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.902524 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 11 16:13:49.902696 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.902525 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 16:13:49.902696 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.902560 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 11 16:13:49.902696 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.902580 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nzxbk\"" Jun 11 16:13:49.902696 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.902560 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 11 16:13:49.902696 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.902557 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 11 16:13:49.903086 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.903034 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 11 16:13:49.903166 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.903156 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a8msgn3nenbih\"" Jun 11 16:13:49.903166 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.903160 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 11 16:13:49.903274 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.903165 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 11 16:13:49.903274 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.903202 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 11 16:13:49.910296 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.910266 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 11 16:13:49.910434 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.910418 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 11 16:13:49.910586 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.910574 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 11 16:13:49.910760 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.910746 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 11 16:13:49.912922 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.912904 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 11 16:13:49.915561 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.915535 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 11 16:13:49.923565 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.923543 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:13:49.953903 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.953875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.953903 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.953903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954065 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.953950 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954065 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.953970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954065 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.953986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954065 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954065 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954215 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954215 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954215 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954215 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954157 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954215 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954206 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954357 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954357 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954248 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954357 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954264 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954357 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbsj8\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-kube-api-access-gbsj8\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954357 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954494 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954494 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.954494 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.954411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:49.987312 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.987255 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" event={"ID":"8741f554-4e17-4c65-84e7-ac9221141bbc","Type":"ContainerStarted","Data":"f92990199e3ce9e7b649e3f10732f8385cb8366fe7ef2592a26f154a044e4de5"} Jun 11 16:13:49.990112 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.990090 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"4ddad168a1ee8976ba24c5181b9cdbe2f2c6ba1e9ed0ff7942cb96908e7da5d4"} Jun 11 16:13:49.990201 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:49.990139 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerStarted","Data":"cace8f7665f2d9a1338bed613fb77d7927b5e6ce7e34412d360d92c8919fd1b5"} Jun 11 16:13:50.023813 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.023774 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.929350768 podStartE2EDuration="6.023759399s" totalImagesPullingTime="4.078401434s" totalInitContainerRuntime="16.007197ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:50.023173154 +0000 UTC m=+294.388527326" watchObservedRunningTime="2026-06-11 16:13:50.023759399 +0000 UTC m=+294.389113569" Jun 11 16:13:50.055753 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.055887 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.055887 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055808 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.055887 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.055887 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.055887 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.055972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056000 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056179 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbsj8\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-kube-api-access-gbsj8\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056421 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056371 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056679 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056427 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056679 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056679 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056481 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.056833 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056703 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.057003 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.056954 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.058024 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.057898 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.058397 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.058354 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.059033 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.059005 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.061840 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.061822 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.062588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.062248 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.062701 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.062609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.063535 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.063390 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:50.063676 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-77sz2" podUID="72260f98-ae30-4e5d-ac3f-989e32672507" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.063862 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.064014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.064014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.064255 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.064420 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.064588 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.064555 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.065120 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.065093 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.066325 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.066283 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.066585 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.066470 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.067307 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.067267 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.073826 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.073791 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbsj8\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-kube-api-access-gbsj8\") pod \"prometheus-k8s-0\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.208782 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.208755 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:13:50.277327 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.277304 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:50.364747 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.364720 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:13:50.367468 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:13:50.367438 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ba35047_ba50_40b4_9249_a8b69cbdd4e4.slice/crio-507685ca73ac3a6ac3601607dc63f988ac564435f3b033a3cad6ace9e2a75aa9 WatchSource:0}: Error finding container 507685ca73ac3a6ac3601607dc63f988ac564435f3b033a3cad6ace9e2a75aa9: Status 404 returned error can't find the container with id 507685ca73ac3a6ac3601607dc63f988ac564435f3b033a3cad6ace9e2a75aa9 Jun 11 16:13:50.679207 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.679130 2560 patch_prober.go:28] interesting pod/image-registry-658f6f668-s6lng container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 16:13:50.679207 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.679178 2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-658f6f668-s6lng" podUID="0101db04-d384-4db3-a0b8-d17758cf3c00" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:13:50.995742 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.995708 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" exitCode=0 Jun 11 16:13:50.996222 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.995794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} Jun 11 16:13:50.996222 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:50.995853 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"507685ca73ac3a6ac3601607dc63f988ac564435f3b033a3cad6ace9e2a75aa9"} Jun 11 16:13:51.943877 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:51.943846 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-z2szh" Jun 11 16:13:52.903219 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:52.903195 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-658f6f668-s6lng" Jun 11 16:13:53.008856 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:53.008771 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" event={"ID":"8741f554-4e17-4c65-84e7-ac9221141bbc","Type":"ContainerStarted","Data":"4971252ddeeae36cdae558f94398a22670dbd24ba189248c99ce868c59eca4a3"} Jun 11 16:13:53.008856 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:53.008835 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" event={"ID":"8741f554-4e17-4c65-84e7-ac9221141bbc","Type":"ContainerStarted","Data":"f961e8d439c971ca66c838507cd85d7ed69b6b1e07f19204d445ec130029f865"} Jun 11 16:13:53.008856 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:53.008861 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" event={"ID":"8741f554-4e17-4c65-84e7-ac9221141bbc","Type":"ContainerStarted","Data":"63d61d6742c79a1f578a81c2d8c8a7da4aa60f09009c6ef82d8621246995c875"} Jun 11 16:13:53.037302 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:53.037206 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9bbf5687c-5cfgj" podStartSLOduration=1.715052546 podStartE2EDuration="5.037190424s" totalImagesPullingTime="3.322137878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:53.034473744 +0000 UTC m=+297.399827916" watchObservedRunningTime="2026-06-11 16:13:53.037190424 +0000 UTC m=+297.402544621" Jun 11 16:13:55.292626 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.292576 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerName="registry" containerID="cri-o://c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11" gracePeriod=30 Jun 11 16:13:55.704194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.704172 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:55.815395 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815362 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-image-registry-private-configuration\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815395 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815400 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-certificates\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815440 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4sbq\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-kube-api-access-l4sbq\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815457 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/201a346a-f445-40f8-8bcf-8e5cb113cc6d-ca-trust-extracted\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815474 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-bound-sa-token\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815539 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-installation-pull-secrets\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815650 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815615 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-trusted-ca\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.815909 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815675 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") pod \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\" (UID: \"201a346a-f445-40f8-8bcf-8e5cb113cc6d\") " Jun 11 16:13:55.816034 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.815983 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:13:55.816149 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.816090 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-certificates\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.817035 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.816781 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:13:55.818755 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.818688 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:13:55.818922 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.818895 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-kube-api-access-l4sbq" (OuterVolumeSpecName: "kube-api-access-l4sbq") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "kube-api-access-l4sbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:13:55.819023 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.818975 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:13:55.819318 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.819295 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:13:55.821062 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.821038 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:13:55.829365 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.829333 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201a346a-f445-40f8-8bcf-8e5cb113cc6d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "201a346a-f445-40f8-8bcf-8e5cb113cc6d" (UID: "201a346a-f445-40f8-8bcf-8e5cb113cc6d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:13:55.917517 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917488 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/201a346a-f445-40f8-8bcf-8e5cb113cc6d-trusted-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.917517 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917514 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-registry-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.917517 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917525 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-image-registry-private-configuration\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.917724 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917535 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4sbq\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-kube-api-access-l4sbq\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.917724 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917545 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/201a346a-f445-40f8-8bcf-8e5cb113cc6d-ca-trust-extracted\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.917724 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917553 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/201a346a-f445-40f8-8bcf-8e5cb113cc6d-bound-sa-token\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:55.917724 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:55.917561 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/201a346a-f445-40f8-8bcf-8e5cb113cc6d-installation-pull-secrets\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:13:56.015969 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.015922 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:13:56.018963 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.018916 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:13:56.024321 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.024296 2560 generic.go:356] "Generic (PLEG): container finished" podID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerID="c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11" exitCode=0 Jun 11 16:13:56.024424 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.024366 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" Jun 11 16:13:56.024424 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.024387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" event={"ID":"201a346a-f445-40f8-8bcf-8e5cb113cc6d","Type":"ContainerDied","Data":"c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11"} Jun 11 16:13:56.024526 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.024466 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cf8c75967-2jd6s" event={"ID":"201a346a-f445-40f8-8bcf-8e5cb113cc6d","Type":"ContainerDied","Data":"e1d9ec7b6c1abaed4ed5ae6b12939fc7c0924133f6656e5a3fdcbd0e67029b8b"} Jun 11 16:13:56.024526 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.024498 2560 scope.go:117] "RemoveContainer" containerID="c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11" Jun 11 16:13:56.027468 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.027444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} Jun 11 16:13:56.027566 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.027510 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} Jun 11 16:13:56.033384 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.033360 2560 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 16:13:56.034733 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.034711 2560 scope.go:117] "RemoveContainer" containerID="c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11" Jun 11 16:13:56.035080 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:13:56.035048 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11\": container with ID starting with c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11 not found: ID does not exist" containerID="c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11" Jun 11 16:13:56.035175 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.035091 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11"} err="failed to get container status \"c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11\": rpc error: code = NotFound desc = could not find container \"c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11\": container with ID starting with c810ee5bcb31627f8dbdbeb483fa8011cd296ef91bc7acb5f3d16037329abe11 not found: ID does not exist" Jun 11 16:13:56.049409 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.049389 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cf8c75967-2jd6s"] Jun 11 16:13:56.054561 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.054458 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cf8c75967-2jd6s"] Jun 11 16:13:56.073882 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:56.066021 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" path="/var/lib/kubelet/pods/201a346a-f445-40f8-8bcf-8e5cb113cc6d/volumes" Jun 11 16:13:59.044208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:59.044156 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} Jun 11 16:13:59.044690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:59.044251 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} Jun 11 16:13:59.044690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:59.044302 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} Jun 11 16:13:59.044690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:59.044344 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerStarted","Data":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} Jun 11 16:13:59.077281 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:13:59.077217 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.048078508 podStartE2EDuration="10.077202656s" totalImagesPullingTime="7.011912076s" totalInitContainerRuntime="17.212072ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:59.075026602 +0000 UTC m=+303.440380773" watchObservedRunningTime="2026-06-11 16:13:59.077202656 +0000 UTC m=+303.442556827" Jun 11 16:14:00.209843 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:00.209812 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:14:02.060226 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:02.060195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:14:10.087823 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:10.087781 2560 generic.go:356] "Generic (PLEG): container finished" podID="8d690600-b47d-4607-a3d7-739b5525f4e4" containerID="c396f53fc88938ecbebebaccf63489c1e56cdbb663421f53cfe3ce2bcec107b0" exitCode=0 Jun 11 16:14:10.088316 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:10.087857 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" event={"ID":"8d690600-b47d-4607-a3d7-739b5525f4e4","Type":"ContainerDied","Data":"c396f53fc88938ecbebebaccf63489c1e56cdbb663421f53cfe3ce2bcec107b0"} Jun 11 16:14:10.088316 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:10.088264 2560 scope.go:117] "RemoveContainer" containerID="c396f53fc88938ecbebebaccf63489c1e56cdbb663421f53cfe3ce2bcec107b0" Jun 11 16:14:11.093827 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:11.093794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-6bc5ff68f5-zkhcz" event={"ID":"8d690600-b47d-4607-a3d7-739b5525f4e4","Type":"ContainerStarted","Data":"fb2919b9d4eef29f23d3fb33dd97db6d6bc763062c584dfd1b30e0916b275e6e"} Jun 11 16:14:25.144894 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:25.144863 2560 generic.go:356] "Generic (PLEG): container finished" podID="559452f3-f51f-44c6-8997-04f31fcae598" containerID="eba10608f240769dadfb53319dbfe989f7897ce4d2dc2405d3368f414c29e57b" exitCode=0 Jun 11 16:14:25.145317 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:25.144954 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-z5kgd" event={"ID":"559452f3-f51f-44c6-8997-04f31fcae598","Type":"ContainerDied","Data":"eba10608f240769dadfb53319dbfe989f7897ce4d2dc2405d3368f414c29e57b"} Jun 11 16:14:25.145387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:25.145359 2560 scope.go:117] "RemoveContainer" containerID="eba10608f240769dadfb53319dbfe989f7897ce4d2dc2405d3368f414c29e57b" Jun 11 16:14:26.150334 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:26.150297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-z5kgd" event={"ID":"559452f3-f51f-44c6-8997-04f31fcae598","Type":"ContainerStarted","Data":"bebbeff14ec46c7b859a33a4ce3e35000458e6eff5b84d91e94f018ee4dc7d97"} Jun 11 16:14:34.182676 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:34.182637 2560 generic.go:356] "Generic (PLEG): container finished" podID="79be130d-d4cc-46d4-a56c-5e216839fcc5" containerID="7f997f9f569623b7930c816822bbb38bff892ff8531a95d116ed5a76c7a456af" exitCode=0 Jun 11 16:14:34.183032 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:34.182710 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" event={"ID":"79be130d-d4cc-46d4-a56c-5e216839fcc5","Type":"ContainerDied","Data":"7f997f9f569623b7930c816822bbb38bff892ff8531a95d116ed5a76c7a456af"} Jun 11 16:14:34.183032 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:34.183025 2560 scope.go:117] "RemoveContainer" containerID="7f997f9f569623b7930c816822bbb38bff892ff8531a95d116ed5a76c7a456af" Jun 11 16:14:35.188028 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:35.187992 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-xv5fk" event={"ID":"79be130d-d4cc-46d4-a56c-5e216839fcc5","Type":"ContainerStarted","Data":"e1eb3587152a969caba9f5327c549cefa5cba87f8b0e1c84c8a01fadb349fd09"} Jun 11 16:14:50.209405 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:50.209321 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:14:50.229666 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:50.229642 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:14:50.267144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:14:50.267118 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:04.123368 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123329 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:15:04.123875 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123683 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="alertmanager" containerID="cri-o://e8525db0484edd6820883d10d1c3e09a26415380a72f71c3ecb39c37c55917fc" gracePeriod=120 Jun 11 16:15:04.123875 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123753 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-metric" containerID="cri-o://cace8f7665f2d9a1338bed613fb77d7927b5e6ce7e34412d360d92c8919fd1b5" gracePeriod=120 Jun 11 16:15:04.123875 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123790 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="prom-label-proxy" containerID="cri-o://4ddad168a1ee8976ba24c5181b9cdbe2f2c6ba1e9ed0ff7942cb96908e7da5d4" gracePeriod=120 Jun 11 16:15:04.123875 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123803 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="config-reloader" containerID="cri-o://13f9f096535b0a4a4b232ff048897461bf389bddd60d713b20e84e31915cc004" gracePeriod=120 Jun 11 16:15:04.123875 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123839 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy" containerID="cri-o://e1235c9397acb24b7bbfe874cb9e27ee3c90d05e870b1e9657470016ffb291c6" gracePeriod=120 Jun 11 16:15:04.124126 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.123794 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-web" containerID="cri-o://03cbdc26df9d47cf29407b20f29470187f184ab2ffa71af54e86c9513d4c1d23" gracePeriod=120 Jun 11 16:15:04.310787 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310762 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="4ddad168a1ee8976ba24c5181b9cdbe2f2c6ba1e9ed0ff7942cb96908e7da5d4" exitCode=0 Jun 11 16:15:04.310787 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310783 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="e1235c9397acb24b7bbfe874cb9e27ee3c90d05e870b1e9657470016ffb291c6" exitCode=0 Jun 11 16:15:04.310787 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310791 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="13f9f096535b0a4a4b232ff048897461bf389bddd60d713b20e84e31915cc004" exitCode=0 Jun 11 16:15:04.311048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310798 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="e8525db0484edd6820883d10d1c3e09a26415380a72f71c3ecb39c37c55917fc" exitCode=0 Jun 11 16:15:04.311048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310828 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"4ddad168a1ee8976ba24c5181b9cdbe2f2c6ba1e9ed0ff7942cb96908e7da5d4"} Jun 11 16:15:04.311048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310898 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"e1235c9397acb24b7bbfe874cb9e27ee3c90d05e870b1e9657470016ffb291c6"} Jun 11 16:15:04.311048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.310962 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"13f9f096535b0a4a4b232ff048897461bf389bddd60d713b20e84e31915cc004"} Jun 11 16:15:04.311048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:04.311003 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"e8525db0484edd6820883d10d1c3e09a26415380a72f71c3ecb39c37c55917fc"} Jun 11 16:15:05.320209 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.320180 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="cace8f7665f2d9a1338bed613fb77d7927b5e6ce7e34412d360d92c8919fd1b5" exitCode=0 Jun 11 16:15:05.320209 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.320208 2560 generic.go:356] "Generic (PLEG): container finished" podID="d574abce-3120-4d7b-bab6-1315f2971c53" containerID="03cbdc26df9d47cf29407b20f29470187f184ab2ffa71af54e86c9513d4c1d23" exitCode=0 Jun 11 16:15:05.320612 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.320292 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"cace8f7665f2d9a1338bed613fb77d7927b5e6ce7e34412d360d92c8919fd1b5"} Jun 11 16:15:05.320612 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.320366 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"03cbdc26df9d47cf29407b20f29470187f184ab2ffa71af54e86c9513d4c1d23"} Jun 11 16:15:05.380723 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.380696 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:05.402189 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402154 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-tls-assets\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.402350 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402241 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-main-tls\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.402350 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402322 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.402350 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402350 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-web-config\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.402529 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402383 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-main-db\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402638 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-metrics-client-ca\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402678 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402727 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-trusted-ca-bundle\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402759 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-cluster-tls-config\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402816 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-config-volume\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.402848 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-config-out\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.403090 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-web\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.403295 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.403446 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-main-db\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.403623 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.403465 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-metrics-client-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.404249 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.403646 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:05.405629 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.405601 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.406201 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.406173 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-config-volume" (OuterVolumeSpecName: "config-volume") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.406306 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.406195 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.406306 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.406177 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:15:05.406403 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.406305 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-config-out" (OuterVolumeSpecName: "config-out") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:15:05.406441 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.406414 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.409834 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.409811 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.418440 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.418418 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-web-config" (OuterVolumeSpecName: "web-config") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.503783 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.503760 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pq89\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-kube-api-access-5pq89\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.503975 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.503898 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy\") pod \"d574abce-3120-4d7b-bab6-1315f2971c53\" (UID: \"d574abce-3120-4d7b-bab6-1315f2971c53\") " Jun 11 16:15:05.504144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504128 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-tls-assets\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504213 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504150 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-main-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504213 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504164 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504213 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504179 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-web-config\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504213 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504194 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d574abce-3120-4d7b-bab6-1315f2971c53-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504213 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504208 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-cluster-tls-config\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504398 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504222 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-config-volume\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504398 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504234 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d574abce-3120-4d7b-bab6-1315f2971c53-config-out\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.504398 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.504248 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.505774 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.505749 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-kube-api-access-5pq89" (OuterVolumeSpecName: "kube-api-access-5pq89") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "kube-api-access-5pq89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:15:05.505830 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.505816 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d574abce-3120-4d7b-bab6-1315f2971c53" (UID: "d574abce-3120-4d7b-bab6-1315f2971c53"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:05.604867 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.604834 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d574abce-3120-4d7b-bab6-1315f2971c53-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:05.604867 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:05.604862 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pq89\" (UniqueName: \"kubernetes.io/projected/d574abce-3120-4d7b-bab6-1315f2971c53-kube-api-access-5pq89\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:06.326641 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.326556 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d574abce-3120-4d7b-bab6-1315f2971c53","Type":"ContainerDied","Data":"a518f4d5e69c1df50679bf24afc6084e44dcc79035d9ba4d61f33db4eedf7a2c"} Jun 11 16:15:06.326641 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.326600 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.326641 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.326628 2560 scope.go:117] "RemoveContainer" containerID="4ddad168a1ee8976ba24c5181b9cdbe2f2c6ba1e9ed0ff7942cb96908e7da5d4" Jun 11 16:15:06.334619 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.334599 2560 scope.go:117] "RemoveContainer" containerID="cace8f7665f2d9a1338bed613fb77d7927b5e6ce7e34412d360d92c8919fd1b5" Jun 11 16:15:06.341638 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.341621 2560 scope.go:117] "RemoveContainer" containerID="e1235c9397acb24b7bbfe874cb9e27ee3c90d05e870b1e9657470016ffb291c6" Jun 11 16:15:06.348299 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.348283 2560 scope.go:117] "RemoveContainer" containerID="03cbdc26df9d47cf29407b20f29470187f184ab2ffa71af54e86c9513d4c1d23" Jun 11 16:15:06.355277 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.355252 2560 scope.go:117] "RemoveContainer" containerID="13f9f096535b0a4a4b232ff048897461bf389bddd60d713b20e84e31915cc004" Jun 11 16:15:06.357162 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.357144 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:15:06.362053 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.362038 2560 scope.go:117] "RemoveContainer" containerID="e8525db0484edd6820883d10d1c3e09a26415380a72f71c3ecb39c37c55917fc" Jun 11 16:15:06.367567 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.367548 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:15:06.371816 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.371796 2560 scope.go:117] "RemoveContainer" containerID="dec5b2ba9c3671af04cc8d2795656f0fad2a13c6192dcc353f0fb29c6591bbe9" Jun 11 16:15:06.407242 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407218 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:15:06.407501 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407490 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="init-config-reloader" Jun 11 16:15:06.407542 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407503 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="init-config-reloader" Jun 11 16:15:06.407542 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407514 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="config-reloader" Jun 11 16:15:06.407542 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407521 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="config-reloader" Jun 11 16:15:06.407542 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407536 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy" Jun 11 16:15:06.407542 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407541 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407550 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-metric" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407556 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-metric" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407562 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerName="registry" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407567 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerName="registry" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407576 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="prom-label-proxy" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407580 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="prom-label-proxy" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407588 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-web" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407593 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-web" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407599 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="alertmanager" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407603 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="alertmanager" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407647 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="alertmanager" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407655 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="201a346a-f445-40f8-8bcf-8e5cb113cc6d" containerName="registry" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407662 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-web" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407667 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407673 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="prom-label-proxy" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407681 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="kube-rbac-proxy-metric" Jun 11 16:15:06.407697 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.407687 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" containerName="config-reloader" Jun 11 16:15:06.412876 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.412861 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.419683 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.419665 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 11 16:15:06.419748 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.419669 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 11 16:15:06.419982 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.419964 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 11 16:15:06.420108 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.419966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 11 16:15:06.422121 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.422082 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 11 16:15:06.422725 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.422711 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lb2mt\"" Jun 11 16:15:06.423389 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.423374 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 11 16:15:06.423456 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.423437 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 11 16:15:06.423523 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.423455 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 11 16:15:06.427981 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.427962 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 11 16:15:06.431220 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.431200 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:15:06.510335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510311 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dphz\" (UniqueName: \"kubernetes.io/projected/d52c2df3-bb5d-4770-849a-99538ffbc888-kube-api-access-4dphz\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.510472 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510342 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-config-volume\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.510472 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510366 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.510472 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510383 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.510955 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510422 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.510955 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d52c2df3-bb5d-4770-849a-99538ffbc888-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511072 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.510965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d52c2df3-bb5d-4770-849a-99538ffbc888-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511072 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.511008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d52c2df3-bb5d-4770-849a-99538ffbc888-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511167 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.511082 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511167 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.511126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52c2df3-bb5d-4770-849a-99538ffbc888-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.511168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d52c2df3-bb5d-4770-849a-99538ffbc888-config-out\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511257 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.511212 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-web-config\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.511347 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.511287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612488 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d52c2df3-bb5d-4770-849a-99538ffbc888-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612488 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d52c2df3-bb5d-4770-849a-99538ffbc888-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612488 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612496 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52c2df3-bb5d-4770-849a-99538ffbc888-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612525 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d52c2df3-bb5d-4770-849a-99538ffbc888-config-out\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-web-config\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612610 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dphz\" (UniqueName: \"kubernetes.io/projected/d52c2df3-bb5d-4770-849a-99538ffbc888-kube-api-access-4dphz\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-config-volume\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.612706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612699 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.613103 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.613103 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.613103 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612778 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d52c2df3-bb5d-4770-849a-99538ffbc888-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.613103 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.612925 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d52c2df3-bb5d-4770-849a-99538ffbc888-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.613363 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.613332 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d52c2df3-bb5d-4770-849a-99538ffbc888-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.613582 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.613522 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d52c2df3-bb5d-4770-849a-99538ffbc888-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616061 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616036 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d52c2df3-bb5d-4770-849a-99538ffbc888-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616207 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616185 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-web-config\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616207 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616194 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d52c2df3-bb5d-4770-849a-99538ffbc888-config-out\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616367 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616342 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616634 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-config-volume\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616718 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616700 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.616888 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.616869 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.617412 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.617385 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.617538 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.617511 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d52c2df3-bb5d-4770-849a-99538ffbc888-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.623146 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.623122 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dphz\" (UniqueName: \"kubernetes.io/projected/d52c2df3-bb5d-4770-849a-99538ffbc888-kube-api-access-4dphz\") pod \"alertmanager-main-0\" (UID: \"d52c2df3-bb5d-4770-849a-99538ffbc888\") " pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.721320 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.721300 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 11 16:15:06.864000 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:06.863840 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 11 16:15:06.866653 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:15:06.866625 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd52c2df3_bb5d_4770_849a_99538ffbc888.slice/crio-9921d518400bb938af85253f84a266184cf6ba9002ea13df99c9979124e8d3b5 WatchSource:0}: Error finding container 9921d518400bb938af85253f84a266184cf6ba9002ea13df99c9979124e8d3b5: Status 404 returned error can't find the container with id 9921d518400bb938af85253f84a266184cf6ba9002ea13df99c9979124e8d3b5 Jun 11 16:15:07.332789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:07.332763 2560 generic.go:356] "Generic (PLEG): container finished" podID="d52c2df3-bb5d-4770-849a-99538ffbc888" containerID="1eda63ecef7c57fd7e9f42cc3fff507f8c28e26819b026321ce2e1bb30d31031" exitCode=0 Jun 11 16:15:07.333158 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:07.332823 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerDied","Data":"1eda63ecef7c57fd7e9f42cc3fff507f8c28e26819b026321ce2e1bb30d31031"} Jun 11 16:15:07.333158 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:07.332870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"9921d518400bb938af85253f84a266184cf6ba9002ea13df99c9979124e8d3b5"} Jun 11 16:15:08.066364 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.066329 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d574abce-3120-4d7b-bab6-1315f2971c53" path="/var/lib/kubelet/pods/d574abce-3120-4d7b-bab6-1315f2971c53/volumes" Jun 11 16:15:08.339585 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.339503 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"54aff170ea35de9fabbb9ffb98e53d8ae4242b0c3c9bf7bb3d5a683e3cf0b2de"} Jun 11 16:15:08.339585 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.339568 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"6ae7f30117ddce0dafaccb29023bdfc28bcb561ef0c4b13680a115e154f7a82a"} Jun 11 16:15:08.339973 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.339594 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"e8234d13359362f0200b08b7286565f23d7ed82a28c977e4e2912cb4e4488108"} Jun 11 16:15:08.339973 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.339618 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"c4b6584d9d28915a2796a7510f59cbc5c9f0169a2dff2e6e9d58eccf482dfcf0"} Jun 11 16:15:08.339973 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.339641 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"6cd23bf8009ec46cc7e24be9366efa24b858c59883cd0e8f91c868607c079f55"} Jun 11 16:15:08.339973 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.339664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d52c2df3-bb5d-4770-849a-99538ffbc888","Type":"ContainerStarted","Data":"ae01d674c4dc37a14168075bba9c5b903930777d2fab032debf55a2fcb062a1d"} Jun 11 16:15:08.383436 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.383382 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.368662786 podStartE2EDuration="2.383366503s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.703717ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:15:08.382748943 +0000 UTC m=+372.748103117" watchObservedRunningTime="2026-06-11 16:15:08.383366503 +0000 UTC m=+372.748720675" Jun 11 16:15:08.476357 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476320 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:15:08.476622 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476584 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="prometheus" containerID="cri-o://d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" gracePeriod=600 Jun 11 16:15:08.476763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476614 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy" containerID="cri-o://9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" gracePeriod=600 Jun 11 16:15:08.476763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476642 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="thanos-sidecar" containerID="cri-o://5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" gracePeriod=600 Jun 11 16:15:08.476763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476660 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-web" containerID="cri-o://d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" gracePeriod=600 Jun 11 16:15:08.476763 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476642 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" gracePeriod=600 Jun 11 16:15:08.476922 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.476646 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="config-reloader" containerID="cri-o://dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" gracePeriod=600 Jun 11 16:15:08.734717 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.734683 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:08.828920 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.828895 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-1\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829101 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.828950 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-metrics-client-certs\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829101 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.828969 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-0\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829222 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829132 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-db\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829222 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829169 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829222 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829207 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbsj8\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-kube-api-access-gbsj8\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829243 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829248 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829278 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-thanos-prometheus-http-client-file\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829317 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-grpc-tls\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829343 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-2\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829375 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-kubelet-serving-ca-bundle\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829407 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-tls\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829448 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-tls-assets\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829473 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-kube-rbac-proxy\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829510 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-trusted-ca-bundle\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829543 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config-out\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829567 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-web-config\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829624 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-metrics-client-ca\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829648 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.829789 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829677 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-serving-certs-ca-bundle\") pod \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\" (UID: \"7ba35047-ba50-40b4-9249-a8b69cbdd4e4\") " Jun 11 16:15:08.830211 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.829992 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.830833 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.830802 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.831047 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.831021 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.831233 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.831205 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.831233 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.831222 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.832507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.832477 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:15:08.832507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.832497 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.832667 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.832558 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.833114 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.833086 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:15:08.833211 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.833133 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.833699 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.833670 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-kube-api-access-gbsj8" (OuterVolumeSpecName: "kube-api-access-gbsj8") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "kube-api-access-gbsj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:15:08.834646 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.834595 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.835239 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.835211 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.835628 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.835606 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:15:08.835721 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.835656 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config" (OuterVolumeSpecName: "config") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.835881 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.835862 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.836398 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.836361 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.836468 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.836450 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.836593 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.836573 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config-out" (OuterVolumeSpecName: "config-out") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:15:08.846793 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.846740 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-web-config" (OuterVolumeSpecName: "web-config") pod "7ba35047-ba50-40b4-9249-a8b69cbdd4e4" (UID: "7ba35047-ba50-40b4-9249-a8b69cbdd4e4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:15:08.931279 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931244 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-tls-assets\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931279 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931278 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-kube-rbac-proxy\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931288 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931297 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config-out\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931305 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-web-config\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931312 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-metrics-client-ca\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931319 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-config\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931327 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931335 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-metrics-client-certs\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931343 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931352 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-db\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931361 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931370 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbsj8\" (UniqueName: \"kubernetes.io/projected/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-kube-api-access-gbsj8\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931380 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931388 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931396 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931397 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-grpc-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931786 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931407 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931786 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931415 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:08.931786 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:08.931423 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ba35047-ba50-40b4-9249-a8b69cbdd4e4-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-161.ec2.internal\" DevicePath \"\"" Jun 11 16:15:09.346535 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346509 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" exitCode=0 Jun 11 16:15:09.346535 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346533 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" exitCode=0 Jun 11 16:15:09.346535 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346540 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" exitCode=0 Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346546 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" exitCode=0 Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346551 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" exitCode=0 Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346556 2560 generic.go:356] "Generic (PLEG): container finished" podID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" exitCode=0 Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346585 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346602 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346725 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346750 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346777 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346787 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346827 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} Jun 11 16:15:09.346998 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.346864 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ba35047-ba50-40b4-9249-a8b69cbdd4e4","Type":"ContainerDied","Data":"507685ca73ac3a6ac3601607dc63f988ac564435f3b033a3cad6ace9e2a75aa9"} Jun 11 16:15:09.357801 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.357763 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.365161 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.365142 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.371797 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.371782 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.377171 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.377144 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:15:09.379144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.379130 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.382601 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.382579 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:15:09.386407 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.386389 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.393617 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.393602 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.400221 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400204 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.400466 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.400446 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.400518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400475 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} err="failed to get container status \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" Jun 11 16:15:09.400518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400492 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.400712 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.400694 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.400748 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400717 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} err="failed to get container status \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" Jun 11 16:15:09.400748 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400732 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.400958 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.400940 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.401004 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400964 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} err="failed to get container status \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" Jun 11 16:15:09.401004 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.400978 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.401215 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.401199 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.401254 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401218 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} err="failed to get container status \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" Jun 11 16:15:09.401254 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401232 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.401430 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.401411 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.401465 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401435 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} err="failed to get container status \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" Jun 11 16:15:09.401465 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401447 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.401630 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.401615 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.401669 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401633 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} err="failed to get container status \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" Jun 11 16:15:09.401669 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401647 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.401819 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:15:09.401801 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.401853 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401822 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} err="failed to get container status \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" Jun 11 16:15:09.401853 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.401833 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.402031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402013 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} err="failed to get container status \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" Jun 11 16:15:09.402088 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402032 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.402246 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402229 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} err="failed to get container status \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" Jun 11 16:15:09.402307 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402246 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.402460 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402439 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} err="failed to get container status \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" Jun 11 16:15:09.402504 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402460 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.402655 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402641 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} err="failed to get container status \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" Jun 11 16:15:09.402706 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402655 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.402853 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402834 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} err="failed to get container status \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" Jun 11 16:15:09.402899 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.402854 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.403095 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403079 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} err="failed to get container status \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" Jun 11 16:15:09.403095 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403094 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.403337 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403318 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} err="failed to get container status \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" Jun 11 16:15:09.403387 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403338 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.403533 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403518 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} err="failed to get container status \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" Jun 11 16:15:09.403586 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403534 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.403750 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403731 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} err="failed to get container status \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" Jun 11 16:15:09.403819 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.403753 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.404025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404008 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} err="failed to get container status \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" Jun 11 16:15:09.404079 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404026 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.404222 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404205 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} err="failed to get container status \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" Jun 11 16:15:09.404263 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404223 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.404440 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404421 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} err="failed to get container status \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" Jun 11 16:15:09.404494 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404440 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.404666 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404650 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} err="failed to get container status \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" Jun 11 16:15:09.404704 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404666 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.404876 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404855 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} err="failed to get container status \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" Jun 11 16:15:09.404916 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.404877 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.405120 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405103 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} err="failed to get container status \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" Jun 11 16:15:09.405172 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405119 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.405313 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405298 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} err="failed to get container status \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" Jun 11 16:15:09.405366 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405314 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.405501 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405482 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} err="failed to get container status \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" Jun 11 16:15:09.405501 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405498 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.405705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405683 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} err="failed to get container status \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" Jun 11 16:15:09.405771 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405706 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.405944 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405911 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} err="failed to get container status \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" Jun 11 16:15:09.406002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.405979 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.406189 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406172 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} err="failed to get container status \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" Jun 11 16:15:09.406260 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406190 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.406397 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406376 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} err="failed to get container status \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" Jun 11 16:15:09.406442 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406399 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.406572 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406555 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} err="failed to get container status \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" Jun 11 16:15:09.406572 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406571 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.406795 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406779 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} err="failed to get container status \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" Jun 11 16:15:09.406867 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.406796 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.407045 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407026 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} err="failed to get container status \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" Jun 11 16:15:09.407096 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407047 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.407261 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407243 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} err="failed to get container status \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" Jun 11 16:15:09.407312 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407260 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.407463 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407446 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} err="failed to get container status \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" Jun 11 16:15:09.407529 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407464 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.407659 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407645 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} err="failed to get container status \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" Jun 11 16:15:09.407705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407659 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.407848 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407832 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} err="failed to get container status \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" Jun 11 16:15:09.407892 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.407849 2560 scope.go:117] "RemoveContainer" containerID="44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1" Jun 11 16:15:09.408075 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408056 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1"} err="failed to get container status \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": rpc error: code = NotFound desc = could not find container \"44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1\": container with ID starting with 44a696bf7c52b689dca157f87b317241c61a522d91bb184c413ef08317628cf1 not found: ID does not exist" Jun 11 16:15:09.408143 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408076 2560 scope.go:117] "RemoveContainer" containerID="9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee" Jun 11 16:15:09.408318 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408300 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee"} err="failed to get container status \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": rpc error: code = NotFound desc = could not find container \"9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee\": container with ID starting with 9b197b6e1a326db21aad971d5021960659ef39dc131b04c228bf927283f7b2ee not found: ID does not exist" Jun 11 16:15:09.408365 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408320 2560 scope.go:117] "RemoveContainer" containerID="d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058" Jun 11 16:15:09.408516 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408500 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058"} err="failed to get container status \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": rpc error: code = NotFound desc = could not find container \"d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058\": container with ID starting with d298d00248b34cf31883f1e85fe05c7ff00a87bb419d54d4e1208c5ca6f65058 not found: ID does not exist" Jun 11 16:15:09.408562 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408526 2560 scope.go:117] "RemoveContainer" containerID="5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3" Jun 11 16:15:09.408704 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408684 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3"} err="failed to get container status \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": rpc error: code = NotFound desc = could not find container \"5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3\": container with ID starting with 5a8a0592d8d7f32c4dd07be5baa53bb4a23ece1ecc9109a89389f743a178cbc3 not found: ID does not exist" Jun 11 16:15:09.408776 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408706 2560 scope.go:117] "RemoveContainer" containerID="dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f" Jun 11 16:15:09.408970 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408947 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f"} err="failed to get container status \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": rpc error: code = NotFound desc = could not find container \"dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f\": container with ID starting with dd3f2eaee51b03e2be5a89ae6366ced34583be94315c8116efe8fa12e02b392f not found: ID does not exist" Jun 11 16:15:09.408970 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.408970 2560 scope.go:117] "RemoveContainer" containerID="d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c" Jun 11 16:15:09.409156 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.409141 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c"} err="failed to get container status \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": rpc error: code = NotFound desc = could not find container \"d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c\": container with ID starting with d3beedbe6515bce8e2c54adda2483a4431705db6002caaeeaae874e797fca41c not found: ID does not exist" Jun 11 16:15:09.409156 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.409155 2560 scope.go:117] "RemoveContainer" containerID="8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f" Jun 11 16:15:09.409315 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.409298 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f"} err="failed to get container status \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": rpc error: code = NotFound desc = could not find container \"8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f\": container with ID starting with 8017be5399332e35e16c02e8aa371c99973fd06b47de74ff96e574007bfb288f not found: ID does not exist" Jun 11 16:15:09.423227 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423202 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:15:09.423509 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423498 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy" Jun 11 16:15:09.423546 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423512 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy" Jun 11 16:15:09.423546 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423524 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="prometheus" Jun 11 16:15:09.423546 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423530 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="prometheus" Jun 11 16:15:09.423546 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423538 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="config-reloader" Jun 11 16:15:09.423546 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423543 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="config-reloader" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423552 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="init-config-reloader" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423558 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="init-config-reloader" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423568 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-web" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423573 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-web" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423579 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="thanos-sidecar" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423584 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="thanos-sidecar" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423589 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-thanos" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423594 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-thanos" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423642 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="prometheus" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423652 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423661 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="config-reloader" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423672 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-web" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423681 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="thanos-sidecar" Jun 11 16:15:09.423687 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.423689 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" containerName="kube-rbac-proxy-thanos" Jun 11 16:15:09.428667 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.428652 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.431462 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.431442 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 11 16:15:09.431462 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.431458 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 11 16:15:09.434064 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d349060-7659-4578-a87c-f73ca77da27c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434081 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434144 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434272 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434251 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434366 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434418 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434461 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-config\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434500 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434541 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434531 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434590 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434566 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434679 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434660 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434751 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434788 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434745 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434838 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434779 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434838 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434947 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434947 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434888 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.434947 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434910 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d349060-7659-4578-a87c-f73ca77da27c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.435110 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.434986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8svj8\" (UniqueName: \"kubernetes.io/projected/9d349060-7659-4578-a87c-f73ca77da27c-kube-api-access-8svj8\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.435110 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.435036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.435293 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.435216 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 11 16:15:09.435348 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.435311 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 11 16:15:09.435680 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.435473 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 11 16:15:09.435680 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.435485 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 11 16:15:09.435955 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.435921 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 11 16:15:09.436125 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436104 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 11 16:15:09.436258 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436104 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a8msgn3nenbih\"" Jun 11 16:15:09.436258 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436143 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 11 16:15:09.436258 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436161 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 11 16:15:09.436258 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436255 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nzxbk\"" Jun 11 16:15:09.436548 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 11 16:15:09.436605 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.436547 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 11 16:15:09.437256 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.437242 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 11 16:15:09.438710 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.438685 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 11 16:15:09.443048 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.443032 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 11 16:15:09.451279 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.451233 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:15:09.536051 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536051 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536048 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536267 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536267 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536267 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536267 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536267 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536221 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.536267 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d349060-7659-4578-a87c-f73ca77da27c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536762 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536788 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8svj8\" (UniqueName: \"kubernetes.io/projected/9d349060-7659-4578-a87c-f73ca77da27c-kube-api-access-8svj8\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536763 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536823 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d349060-7659-4578-a87c-f73ca77da27c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536883 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.536911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537002 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537080 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-config\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.537335 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.537319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.539869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.538566 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.539869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.539004 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.539869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.539468 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.539869 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.539732 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.540675 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.539953 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.540675 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.540290 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9d349060-7659-4578-a87c-f73ca77da27c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.540675 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.540519 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.541020 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.540972 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9d349060-7659-4578-a87c-f73ca77da27c-config-out\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.541337 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.541283 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9d349060-7659-4578-a87c-f73ca77da27c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.541440 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.541378 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.542067 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.542050 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.542138 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.542083 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.542781 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.542754 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-web-config\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.542860 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.542818 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.543263 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.543247 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9d349060-7659-4578-a87c-f73ca77da27c-config\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.547177 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.547160 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8svj8\" (UniqueName: \"kubernetes.io/projected/9d349060-7659-4578-a87c-f73ca77da27c-kube-api-access-8svj8\") pod \"prometheus-k8s-0\" (UID: \"9d349060-7659-4578-a87c-f73ca77da27c\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.738766 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.738741 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:15:09.880972 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.880944 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 11 16:15:09.886197 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:15:09.886163 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d349060_7659_4578_a87c_f73ca77da27c.slice/crio-403f06f2eafaa506204888834123698dcb23352de4a31509f971f328e33a781b WatchSource:0}: Error finding container 403f06f2eafaa506204888834123698dcb23352de4a31509f971f328e33a781b: Status 404 returned error can't find the container with id 403f06f2eafaa506204888834123698dcb23352de4a31509f971f328e33a781b Jun 11 16:15:09.941009 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.940986 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:15:09.943326 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:09.943306 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/72260f98-ae30-4e5d-ac3f-989e32672507-metrics-certs\") pod \"network-metrics-daemon-77sz2\" (UID: \"72260f98-ae30-4e5d-ac3f-989e32672507\") " pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:15:10.065618 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.065547 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ba35047-ba50-40b4-9249-a8b69cbdd4e4" path="/var/lib/kubelet/pods/7ba35047-ba50-40b4-9249-a8b69cbdd4e4/volumes" Jun 11 16:15:10.164108 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.164083 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kx8mw\"" Jun 11 16:15:10.172354 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.172334 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-77sz2" Jun 11 16:15:10.306897 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.306875 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-77sz2"] Jun 11 16:15:10.308909 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:15:10.308882 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72260f98_ae30_4e5d_ac3f_989e32672507.slice/crio-d869bfccfdb31d3bbdbd0867237a9ba8b477d51f2c2b3b8776b47fbb1bf78baf WatchSource:0}: Error finding container d869bfccfdb31d3bbdbd0867237a9ba8b477d51f2c2b3b8776b47fbb1bf78baf: Status 404 returned error can't find the container with id d869bfccfdb31d3bbdbd0867237a9ba8b477d51f2c2b3b8776b47fbb1bf78baf Jun 11 16:15:10.311062 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.311048 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:15:10.350966 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.350887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-77sz2" event={"ID":"72260f98-ae30-4e5d-ac3f-989e32672507","Type":"ContainerStarted","Data":"d869bfccfdb31d3bbdbd0867237a9ba8b477d51f2c2b3b8776b47fbb1bf78baf"} Jun 11 16:15:10.353041 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.353019 2560 generic.go:356] "Generic (PLEG): container finished" podID="9d349060-7659-4578-a87c-f73ca77da27c" containerID="721ad25a8faf181992db040da069d338dd0d86aac93fc04cd3d3c8bd92048ef3" exitCode=0 Jun 11 16:15:10.353116 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.353085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerDied","Data":"721ad25a8faf181992db040da069d338dd0d86aac93fc04cd3d3c8bd92048ef3"} Jun 11 16:15:10.353154 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:10.353129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"403f06f2eafaa506204888834123698dcb23352de4a31509f971f328e33a781b"} Jun 11 16:15:11.362730 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.362679 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"157cd840cb90b8b48edce3a840bab02c46780af1ed5ef2a63be579793997da67"} Jun 11 16:15:11.363208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.362770 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"c81db3d1ae2f88ac8d3cddb8a63600fa8580a49f69230b7a3c66f96f55aeb7d3"} Jun 11 16:15:11.363208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.362803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"9847b412cbbe4354b1456a86aea19eb4ade325d44419cfb4396b7891bef6dedb"} Jun 11 16:15:11.363208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.362839 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"44b90645bdf028ad3d5e56a8666c86d9445f34f912d869bec1cc22c279591742"} Jun 11 16:15:11.363208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.362878 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"b19ad91ba472f28647dd6201e2592d0240882665699cd10d32f72a0fda3cc205"} Jun 11 16:15:11.363208 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.362907 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9d349060-7659-4578-a87c-f73ca77da27c","Type":"ContainerStarted","Data":"9686001f37d12d3d852e4ac63b7320f31f7bf24892d76453e5ec7351dcef9a00"} Jun 11 16:15:11.400722 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:11.400657 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.381112304 podStartE2EDuration="2.400637386s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.525082ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:15:11.398105784 +0000 UTC m=+375.763459957" watchObservedRunningTime="2026-06-11 16:15:11.400637386 +0000 UTC m=+375.765991559" Jun 11 16:15:12.368890 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:12.368852 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-77sz2" event={"ID":"72260f98-ae30-4e5d-ac3f-989e32672507","Type":"ContainerStarted","Data":"c3cf0810a682946e197070eccca65fc2d73084ac8f7ea49227eb721b65821658"} Jun 11 16:15:12.369323 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:12.368922 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-77sz2" event={"ID":"72260f98-ae30-4e5d-ac3f-989e32672507","Type":"ContainerStarted","Data":"801a709357ce2c51e1c6a479bbabda94e8fa4600d5d9ef088ea928e6aa581282"} Jun 11 16:15:12.386651 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:12.386604 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-77sz2" podStartSLOduration=375.162980274 podStartE2EDuration="6m16.386590006s" totalImagesPullingTime="1.223609732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:15:12.384311661 +0000 UTC m=+376.749665831" watchObservedRunningTime="2026-06-11 16:15:12.386590006 +0000 UTC m=+376.751944177" Jun 11 16:15:14.739104 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:15:14.739070 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:16:09.739805 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:16:09.739766 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:16:09.756081 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:16:09.756056 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:16:10.622020 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:16:10.621987 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 11 16:18:14.895238 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:14.895198 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2"] Jun 11 16:18:14.898517 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:14.898500 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:14.902495 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:14.902467 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 11 16:18:14.902949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:14.902907 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-wg7g9\"" Jun 11 16:18:14.903441 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:14.903423 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:18:14.909347 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:14.909327 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2"] Jun 11 16:18:15.041977 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.041915 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36be49fb-ffac-4080-9c77-633f4ac8fd58-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mhvn2\" (UID: \"36be49fb-ffac-4080-9c77-633f4ac8fd58\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.041977 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.041990 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgrgf\" (UniqueName: \"kubernetes.io/projected/36be49fb-ffac-4080-9c77-633f4ac8fd58-kube-api-access-lgrgf\") pod \"openshift-lws-operator-fd8ccff4c-mhvn2\" (UID: \"36be49fb-ffac-4080-9c77-633f4ac8fd58\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.142782 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.142748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgrgf\" (UniqueName: \"kubernetes.io/projected/36be49fb-ffac-4080-9c77-633f4ac8fd58-kube-api-access-lgrgf\") pod \"openshift-lws-operator-fd8ccff4c-mhvn2\" (UID: \"36be49fb-ffac-4080-9c77-633f4ac8fd58\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.142973 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.142849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36be49fb-ffac-4080-9c77-633f4ac8fd58-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mhvn2\" (UID: \"36be49fb-ffac-4080-9c77-633f4ac8fd58\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.143245 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.143223 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36be49fb-ffac-4080-9c77-633f4ac8fd58-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mhvn2\" (UID: \"36be49fb-ffac-4080-9c77-633f4ac8fd58\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.151896 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.151840 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgrgf\" (UniqueName: \"kubernetes.io/projected/36be49fb-ffac-4080-9c77-633f4ac8fd58-kube-api-access-lgrgf\") pod \"openshift-lws-operator-fd8ccff4c-mhvn2\" (UID: \"36be49fb-ffac-4080-9c77-633f4ac8fd58\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.218602 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.218579 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" Jun 11 16:18:15.344767 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:15.344698 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2"] Jun 11 16:18:15.347285 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:18:15.347257 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36be49fb_ffac_4080_9c77_633f4ac8fd58.slice/crio-8c0ba4fbda4bacc875c4bc77979eb7cf76bdcbf20f19d4caf43cfd17739460a2 WatchSource:0}: Error finding container 8c0ba4fbda4bacc875c4bc77979eb7cf76bdcbf20f19d4caf43cfd17739460a2: Status 404 returned error can't find the container with id 8c0ba4fbda4bacc875c4bc77979eb7cf76bdcbf20f19d4caf43cfd17739460a2 Jun 11 16:18:16.103307 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:16.103265 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" event={"ID":"36be49fb-ffac-4080-9c77-633f4ac8fd58","Type":"ContainerStarted","Data":"8c0ba4fbda4bacc875c4bc77979eb7cf76bdcbf20f19d4caf43cfd17739460a2"} Jun 11 16:18:18.113755 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:18.113714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" event={"ID":"36be49fb-ffac-4080-9c77-633f4ac8fd58","Type":"ContainerStarted","Data":"7174e8d5736ae4fa2a1cc735daf428bf71e8e862b8a3a453e3c642588538d8f7"} Jun 11 16:18:18.136344 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:18.136261 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mhvn2" podStartSLOduration=1.606656567 podStartE2EDuration="4.136245972s" totalImagesPullingTime="2.529589405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:18:18.134746897 +0000 UTC m=+562.500101071" watchObservedRunningTime="2026-06-11 16:18:18.136245972 +0000 UTC m=+562.501600143" Jun 11 16:18:36.078787 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.078756 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w"] Jun 11 16:18:36.084402 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.084377 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.089009 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.088989 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 16:18:36.090139 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.090119 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 16:18:36.095165 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.095143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-clg2l\"" Jun 11 16:18:36.095690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.095674 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 16:18:36.095741 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.095686 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 16:18:36.117545 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.117522 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w"] Jun 11 16:18:36.215801 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.215768 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4963d659-4217-4cf0-8c8f-379d2724386b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.215801 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.215803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gnnz\" (UniqueName: \"kubernetes.io/projected/4963d659-4217-4cf0-8c8f-379d2724386b-kube-api-access-2gnnz\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.216029 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.215895 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4963d659-4217-4cf0-8c8f-379d2724386b-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.216029 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.215956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4963d659-4217-4cf0-8c8f-379d2724386b-manifests\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.216029 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.215999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4963d659-4217-4cf0-8c8f-379d2724386b-tmp\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.316892 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.316861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4963d659-4217-4cf0-8c8f-379d2724386b-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.316892 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.316905 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4963d659-4217-4cf0-8c8f-379d2724386b-manifests\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.317153 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.316946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4963d659-4217-4cf0-8c8f-379d2724386b-tmp\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.317153 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.316968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4963d659-4217-4cf0-8c8f-379d2724386b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.317153 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.317099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gnnz\" (UniqueName: \"kubernetes.io/projected/4963d659-4217-4cf0-8c8f-379d2724386b-kube-api-access-2gnnz\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.317292 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.317257 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4963d659-4217-4cf0-8c8f-379d2724386b-manifests\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.319457 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.319425 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4963d659-4217-4cf0-8c8f-379d2724386b-tmp\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.319572 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.319474 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4963d659-4217-4cf0-8c8f-379d2724386b-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.319572 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.319504 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4963d659-4217-4cf0-8c8f-379d2724386b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.326602 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.326579 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gnnz\" (UniqueName: \"kubernetes.io/projected/4963d659-4217-4cf0-8c8f-379d2724386b-kube-api-access-2gnnz\") pod \"opendatahub-operator-controller-manager-fb875574b-z558w\" (UID: \"4963d659-4217-4cf0-8c8f-379d2724386b\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.396150 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.396086 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:36.554827 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:36.554803 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w"] Jun 11 16:18:36.557509 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:18:36.557483 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4963d659_4217_4cf0_8c8f_379d2724386b.slice/crio-e3201240fc8fcebebcc426e578d811c580db93ceff678c978a956e5b1f857f47 WatchSource:0}: Error finding container e3201240fc8fcebebcc426e578d811c580db93ceff678c978a956e5b1f857f47: Status 404 returned error can't find the container with id e3201240fc8fcebebcc426e578d811c580db93ceff678c978a956e5b1f857f47 Jun 11 16:18:37.195173 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:37.195136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" event={"ID":"4963d659-4217-4cf0-8c8f-379d2724386b","Type":"ContainerStarted","Data":"e3201240fc8fcebebcc426e578d811c580db93ceff678c978a956e5b1f857f47"} Jun 11 16:18:40.210307 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:40.210269 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" event={"ID":"4963d659-4217-4cf0-8c8f-379d2724386b","Type":"ContainerStarted","Data":"c499685ca6812069b36fc57f51d60a1411e8495280a700d0ee7d72fca03c8dc1"} Jun 11 16:18:41.216116 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:41.216032 2560 generic.go:356] "Generic (PLEG): container finished" podID="4963d659-4217-4cf0-8c8f-379d2724386b" containerID="c499685ca6812069b36fc57f51d60a1411e8495280a700d0ee7d72fca03c8dc1" exitCode=0 Jun 11 16:18:41.216456 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:41.216123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" event={"ID":"4963d659-4217-4cf0-8c8f-379d2724386b","Type":"ContainerDied","Data":"c499685ca6812069b36fc57f51d60a1411e8495280a700d0ee7d72fca03c8dc1"} Jun 11 16:18:42.224372 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:42.224335 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" event={"ID":"4963d659-4217-4cf0-8c8f-379d2724386b","Type":"ContainerStarted","Data":"300194cb8c1d354998e2fad37dff97c704202869e8919a8aa6fa1ce3c63b4d54"} Jun 11 16:18:42.224859 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:42.224504 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:42.251769 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:42.251730 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" podStartSLOduration=1.791355672 podStartE2EDuration="6.251716297s" totalImagesPullingTime="3.404970468s" totalInitContainerRuntime="1.055390157s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:18:42.250595203 +0000 UTC m=+586.615949387" watchObservedRunningTime="2026-06-11 16:18:42.251716297 +0000 UTC m=+586.617070469" Jun 11 16:18:53.232508 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.232469 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-z558w" Jun 11 16:18:53.540248 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.540216 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx"] Jun 11 16:18:53.547371 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.547349 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.550757 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.550734 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 11 16:18:53.550890 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.550761 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 11 16:18:53.551580 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.551528 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 11 16:18:53.551684 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.551581 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8nvk8\"" Jun 11 16:18:53.560619 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.560600 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx"] Jun 11 16:18:53.566188 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.566165 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5dxz\" (UniqueName: \"kubernetes.io/projected/abee4d4e-9369-422a-a771-cc5f1a567541-kube-api-access-b5dxz\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.566272 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.566215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/abee4d4e-9369-422a-a771-cc5f1a567541-metrics-cert\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.566327 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.566302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/abee4d4e-9369-422a-a771-cc5f1a567541-manager-config\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.566364 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.566341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abee4d4e-9369-422a-a771-cc5f1a567541-cert\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.667740 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.667712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/abee4d4e-9369-422a-a771-cc5f1a567541-manager-config\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.667740 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.667743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abee4d4e-9369-422a-a771-cc5f1a567541-cert\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.667918 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.667799 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5dxz\" (UniqueName: \"kubernetes.io/projected/abee4d4e-9369-422a-a771-cc5f1a567541-kube-api-access-b5dxz\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.667918 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.667858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/abee4d4e-9369-422a-a771-cc5f1a567541-metrics-cert\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.668486 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.668468 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/abee4d4e-9369-422a-a771-cc5f1a567541-manager-config\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.670392 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.670376 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/abee4d4e-9369-422a-a771-cc5f1a567541-metrics-cert\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.670443 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.670419 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abee4d4e-9369-422a-a771-cc5f1a567541-cert\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.678987 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.678963 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5dxz\" (UniqueName: \"kubernetes.io/projected/abee4d4e-9369-422a-a771-cc5f1a567541-kube-api-access-b5dxz\") pod \"lws-controller-manager-67c4b4c7b5-kh9mx\" (UID: \"abee4d4e-9369-422a-a771-cc5f1a567541\") " pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.856820 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.856725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:53.991080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:53.991053 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx"] Jun 11 16:18:53.993533 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:18:53.993507 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabee4d4e_9369_422a_a771_cc5f1a567541.slice/crio-f290fbd8c03e52a204d7c2c734988313885fa75bb9139ad60b8070f42cc97061 WatchSource:0}: Error finding container f290fbd8c03e52a204d7c2c734988313885fa75bb9139ad60b8070f42cc97061: Status 404 returned error can't find the container with id f290fbd8c03e52a204d7c2c734988313885fa75bb9139ad60b8070f42cc97061 Jun 11 16:18:54.277444 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:54.277404 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" event={"ID":"abee4d4e-9369-422a-a771-cc5f1a567541","Type":"ContainerStarted","Data":"f290fbd8c03e52a204d7c2c734988313885fa75bb9139ad60b8070f42cc97061"} Jun 11 16:18:55.947855 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.947823 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms"] Jun 11 16:18:55.951239 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.951208 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:55.954311 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.954287 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 16:18:55.954601 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.954581 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 16:18:55.954700 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.954628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 16:18:55.955705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.955653 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m7zc6\"" Jun 11 16:18:55.955705 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.955660 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 16:18:55.975630 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.975378 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms"] Jun 11 16:18:55.986666 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.986640 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee854e50-564a-4b1d-8368-193fde514639-tls-certs\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:55.986784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.986686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee854e50-564a-4b1d-8368-193fde514639-tmp\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:55.986784 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:55.986770 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm8wl\" (UniqueName: \"kubernetes.io/projected/ee854e50-564a-4b1d-8368-193fde514639-kube-api-access-wm8wl\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.058555 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.058485 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:18:56.070284 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.058689 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:18:56.087665 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.087637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee854e50-564a-4b1d-8368-193fde514639-tls-certs\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.087799 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.087672 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee854e50-564a-4b1d-8368-193fde514639-tmp\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.087862 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.087843 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm8wl\" (UniqueName: \"kubernetes.io/projected/ee854e50-564a-4b1d-8368-193fde514639-kube-api-access-wm8wl\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.090106 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.090079 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee854e50-564a-4b1d-8368-193fde514639-tmp\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.091102 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.091084 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 16:18:56.100354 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.100331 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee854e50-564a-4b1d-8368-193fde514639-tls-certs\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.100446 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.100377 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 16:18:56.109554 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.109537 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 16:18:56.120034 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.120011 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm8wl\" (UniqueName: \"kubernetes.io/projected/ee854e50-564a-4b1d-8368-193fde514639-kube-api-access-wm8wl\") pod \"kube-auth-proxy-6bbbf84858-g96ms\" (UID: \"ee854e50-564a-4b1d-8368-193fde514639\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.267534 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.267506 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m7zc6\"" Jun 11 16:18:56.275359 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.275333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" Jun 11 16:18:56.291880 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.291837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" event={"ID":"abee4d4e-9369-422a-a771-cc5f1a567541","Type":"ContainerStarted","Data":"2827534c62d16f8e3a1c833539ffe6dcf78a1ed7c380d1aabea96bb694e25271"} Jun 11 16:18:56.292025 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.291985 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:18:56.320311 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.320257 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" podStartSLOduration=1.351487585 podStartE2EDuration="3.320239097s" totalImagesPullingTime="1.968751512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:18:56.316335787 +0000 UTC m=+600.681689959" watchObservedRunningTime="2026-06-11 16:18:56.320239097 +0000 UTC m=+600.685593268" Jun 11 16:18:56.424909 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:56.424886 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms"] Jun 11 16:18:56.426866 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:18:56.426838 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee854e50_564a_4b1d_8368_193fde514639.slice/crio-9ce160c35f2e47c5e77ddaf6fdc32b37d3fd86168ac2a3d910fd6cc377e79ed7 WatchSource:0}: Error finding container 9ce160c35f2e47c5e77ddaf6fdc32b37d3fd86168ac2a3d910fd6cc377e79ed7: Status 404 returned error can't find the container with id 9ce160c35f2e47c5e77ddaf6fdc32b37d3fd86168ac2a3d910fd6cc377e79ed7 Jun 11 16:18:57.298510 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:57.298477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" event={"ID":"ee854e50-564a-4b1d-8368-193fde514639","Type":"ContainerStarted","Data":"9ce160c35f2e47c5e77ddaf6fdc32b37d3fd86168ac2a3d910fd6cc377e79ed7"} Jun 11 16:18:59.301250 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:18:59.301228 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 16:19:00.314162 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:19:00.314123 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" event={"ID":"ee854e50-564a-4b1d-8368-193fde514639","Type":"ContainerStarted","Data":"5f9864e9e0ea4321482bb5eb9c02b0c98c6cae0dfe0cfbf9dcdab5c874fdc5f7"} Jun 11 16:19:00.332407 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:19:00.332354 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-g96ms" podStartSLOduration=2.462663125 podStartE2EDuration="5.332334845s" totalImagesPullingTime="2.86967172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:00.332170437 +0000 UTC m=+604.697524612" watchObservedRunningTime="2026-06-11 16:19:00.332334845 +0000 UTC m=+604.697689018" Jun 11 16:19:07.301747 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:19:07.301710 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-67c4b4c7b5-kh9mx" Jun 11 16:20:44.545327 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.545247 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s"] Jun 11 16:20:44.548507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.548491 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.551516 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.551481 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 11 16:20:44.551516 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.551498 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 16:20:44.551690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.551626 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tx77h\"" Jun 11 16:20:44.552534 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.552511 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 16:20:44.552620 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.552512 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 11 16:20:44.567818 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.567794 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s"] Jun 11 16:20:44.608535 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.608509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn6zb\" (UniqueName: \"kubernetes.io/projected/e360d1af-44e6-4dff-a1e4-2406d73392e7-kube-api-access-dn6zb\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.608643 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.608572 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e360d1af-44e6-4dff-a1e4-2406d73392e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.608643 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.608630 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e360d1af-44e6-4dff-a1e4-2406d73392e7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.709730 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.709698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e360d1af-44e6-4dff-a1e4-2406d73392e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.709893 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.709760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e360d1af-44e6-4dff-a1e4-2406d73392e7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.709893 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.709822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn6zb\" (UniqueName: \"kubernetes.io/projected/e360d1af-44e6-4dff-a1e4-2406d73392e7-kube-api-access-dn6zb\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.709893 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:20:44.709856 2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 11 16:20:44.710107 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:20:44.709963 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e360d1af-44e6-4dff-a1e4-2406d73392e7-plugin-serving-cert podName:e360d1af-44e6-4dff-a1e4-2406d73392e7 nodeName:}" failed. No retries permitted until 2026-06-11 16:20:45.209906986 +0000 UTC m=+709.575261136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e360d1af-44e6-4dff-a1e4-2406d73392e7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-gd54s" (UID: "e360d1af-44e6-4dff-a1e4-2406d73392e7") : secret "plugin-serving-cert" not found Jun 11 16:20:44.710489 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.710470 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e360d1af-44e6-4dff-a1e4-2406d73392e7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:44.737308 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:44.737284 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn6zb\" (UniqueName: \"kubernetes.io/projected/e360d1af-44e6-4dff-a1e4-2406d73392e7-kube-api-access-dn6zb\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:45.213340 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:45.213309 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e360d1af-44e6-4dff-a1e4-2406d73392e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:45.215733 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:45.215707 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e360d1af-44e6-4dff-a1e4-2406d73392e7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gd54s\" (UID: \"e360d1af-44e6-4dff-a1e4-2406d73392e7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:45.458850 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:45.458816 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" Jun 11 16:20:45.591331 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:45.591306 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s"] Jun 11 16:20:45.593301 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:20:45.593274 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode360d1af_44e6_4dff_a1e4_2406d73392e7.slice/crio-4c7e24f20a29e7cef63881551433ce107945c0a9f5af25a525f4d0f93cf9dfde WatchSource:0}: Error finding container 4c7e24f20a29e7cef63881551433ce107945c0a9f5af25a525f4d0f93cf9dfde: Status 404 returned error can't find the container with id 4c7e24f20a29e7cef63881551433ce107945c0a9f5af25a525f4d0f93cf9dfde Jun 11 16:20:45.594665 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:45.594648 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:20:45.798151 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:20:45.798121 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" event={"ID":"e360d1af-44e6-4dff-a1e4-2406d73392e7","Type":"ContainerStarted","Data":"4c7e24f20a29e7cef63881551433ce107945c0a9f5af25a525f4d0f93cf9dfde"} Jun 11 16:21:11.946508 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:11.946470 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" event={"ID":"e360d1af-44e6-4dff-a1e4-2406d73392e7","Type":"ContainerStarted","Data":"be9755c69be847e62df9aba6a7b5f2b0debdc4c13a02d22a738353e493e545c8"} Jun 11 16:21:11.965286 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:11.965240 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gd54s" podStartSLOduration=2.410517343 podStartE2EDuration="27.965224396s" totalImagesPullingTime="25.554707053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:21:11.962659292 +0000 UTC m=+736.328013465" watchObservedRunningTime="2026-06-11 16:21:11.965224396 +0000 UTC m=+736.330578568" Jun 11 16:21:25.967550 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:25.967513 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:21:25.970078 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:25.970060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:25.973069 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:25.973049 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 11 16:21:25.983628 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:25.983606 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:21:26.016457 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.016432 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:21:26.077772 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.077734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/63545728-15ed-4587-b1bf-c1448f8986cf-config-file\") pod \"limitador-limitador-54d6d988d5-xhdft\" (UID: \"63545728-15ed-4587-b1bf-c1448f8986cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.077943 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.077799 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kg47\" (UniqueName: \"kubernetes.io/projected/63545728-15ed-4587-b1bf-c1448f8986cf-kube-api-access-8kg47\") pod \"limitador-limitador-54d6d988d5-xhdft\" (UID: \"63545728-15ed-4587-b1bf-c1448f8986cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.179153 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.179120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/63545728-15ed-4587-b1bf-c1448f8986cf-config-file\") pod \"limitador-limitador-54d6d988d5-xhdft\" (UID: \"63545728-15ed-4587-b1bf-c1448f8986cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.179343 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.179162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kg47\" (UniqueName: \"kubernetes.io/projected/63545728-15ed-4587-b1bf-c1448f8986cf-kube-api-access-8kg47\") pod \"limitador-limitador-54d6d988d5-xhdft\" (UID: \"63545728-15ed-4587-b1bf-c1448f8986cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.179755 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.179732 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/63545728-15ed-4587-b1bf-c1448f8986cf-config-file\") pod \"limitador-limitador-54d6d988d5-xhdft\" (UID: \"63545728-15ed-4587-b1bf-c1448f8986cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.193614 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.193585 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kg47\" (UniqueName: \"kubernetes.io/projected/63545728-15ed-4587-b1bf-c1448f8986cf-kube-api-access-8kg47\") pod \"limitador-limitador-54d6d988d5-xhdft\" (UID: \"63545728-15ed-4587-b1bf-c1448f8986cf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.279820 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.279793 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:26.442130 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:26.442103 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:21:26.443649 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:21:26.443623 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63545728_15ed_4587_b1bf_c1448f8986cf.slice/crio-1fcc24a1dd95b17fb7060bd4815c20e4542e51e958e46040c73f93b29b0870ba WatchSource:0}: Error finding container 1fcc24a1dd95b17fb7060bd4815c20e4542e51e958e46040c73f93b29b0870ba: Status 404 returned error can't find the container with id 1fcc24a1dd95b17fb7060bd4815c20e4542e51e958e46040c73f93b29b0870ba Jun 11 16:21:27.018664 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:27.018557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" event={"ID":"63545728-15ed-4587-b1bf-c1448f8986cf","Type":"ContainerStarted","Data":"1fcc24a1dd95b17fb7060bd4815c20e4542e51e958e46040c73f93b29b0870ba"} Jun 11 16:21:29.030320 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:29.030283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" event={"ID":"63545728-15ed-4587-b1bf-c1448f8986cf","Type":"ContainerStarted","Data":"494e1466d4ce0fa76e287572b4608c0e2f038fdf156aa7976383f7be2d489271"} Jun 11 16:21:29.030708 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:29.030461 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:21:29.051056 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:29.050980 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" podStartSLOduration=1.709302611 podStartE2EDuration="4.050966184s" totalImagesPullingTime="2.341663573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:21:29.049578367 +0000 UTC m=+753.414932540" watchObservedRunningTime="2026-06-11 16:21:29.050966184 +0000 UTC m=+753.416320378" Jun 11 16:21:40.036243 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:21:40.036215 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xhdft" Jun 11 16:22:59.029963 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.029911 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-766f66d7b8-t94xt"] Jun 11 16:22:59.032341 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.032322 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.038134 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.038111 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 11 16:22:59.038692 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.038674 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sv65v\"" Jun 11 16:22:59.039130 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.039115 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 11 16:22:59.049781 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.049753 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-766f66d7b8-t94xt"] Jun 11 16:22:59.109056 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.109027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103d0c3b-7aa8-4351-a0a8-044447e89cf4-webhook-cert\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.109225 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.109073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbsvc\" (UniqueName: \"kubernetes.io/projected/103d0c3b-7aa8-4351-a0a8-044447e89cf4-kube-api-access-zbsvc\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.210548 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.210501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103d0c3b-7aa8-4351-a0a8-044447e89cf4-webhook-cert\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.210751 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.210562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbsvc\" (UniqueName: \"kubernetes.io/projected/103d0c3b-7aa8-4351-a0a8-044447e89cf4-kube-api-access-zbsvc\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.210751 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:22:59.210675 2560 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 11 16:22:59.210751 ip-10-0-128-161 kubenswrapper[2560]: E0611 16:22:59.210749 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/103d0c3b-7aa8-4351-a0a8-044447e89cf4-webhook-cert podName:103d0c3b-7aa8-4351-a0a8-044447e89cf4 nodeName:}" failed. No retries permitted until 2026-06-11 16:22:59.710731275 +0000 UTC m=+844.076085425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/103d0c3b-7aa8-4351-a0a8-044447e89cf4-webhook-cert") pod "maas-controller-766f66d7b8-t94xt" (UID: "103d0c3b-7aa8-4351-a0a8-044447e89cf4") : secret "maas-controller-webhook-cert" not found Jun 11 16:22:59.221825 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.221791 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbsvc\" (UniqueName: \"kubernetes.io/projected/103d0c3b-7aa8-4351-a0a8-044447e89cf4-kube-api-access-zbsvc\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.715192 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.715151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103d0c3b-7aa8-4351-a0a8-044447e89cf4-webhook-cert\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.717776 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.717756 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/103d0c3b-7aa8-4351-a0a8-044447e89cf4-webhook-cert\") pod \"maas-controller-766f66d7b8-t94xt\" (UID: \"103d0c3b-7aa8-4351-a0a8-044447e89cf4\") " pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:22:59.943685 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:22:59.943645 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:23:00.083944 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:00.083906 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-766f66d7b8-t94xt"] Jun 11 16:23:00.086327 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:23:00.086302 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103d0c3b_7aa8_4351_a0a8_044447e89cf4.slice/crio-fbbd7d5cfe379cb70a492fc5151ef2c6fbdaf5338fce36fb1ddcea2d35e4ce61 WatchSource:0}: Error finding container fbbd7d5cfe379cb70a492fc5151ef2c6fbdaf5338fce36fb1ddcea2d35e4ce61: Status 404 returned error can't find the container with id fbbd7d5cfe379cb70a492fc5151ef2c6fbdaf5338fce36fb1ddcea2d35e4ce61 Jun 11 16:23:00.453123 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:00.453026 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-766f66d7b8-t94xt" event={"ID":"103d0c3b-7aa8-4351-a0a8-044447e89cf4","Type":"ContainerStarted","Data":"fbbd7d5cfe379cb70a492fc5151ef2c6fbdaf5338fce36fb1ddcea2d35e4ce61"} Jun 11 16:23:02.465119 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:02.465085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-766f66d7b8-t94xt" event={"ID":"103d0c3b-7aa8-4351-a0a8-044447e89cf4","Type":"ContainerStarted","Data":"eac10dd137892c419c96b36a28fa28fd766d523ad410387822f5c4a06f1a0b33"} Jun 11 16:23:02.465558 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:02.465230 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:23:02.489417 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:02.489373 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-766f66d7b8-t94xt" podStartSLOduration=1.202816793 podStartE2EDuration="3.489357423s" totalImagesPullingTime="2.28654063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:22:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:23:02.485989003 +0000 UTC m=+846.851343175" watchObservedRunningTime="2026-06-11 16:23:02.489357423 +0000 UTC m=+846.854711594" Jun 11 16:23:13.475727 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:13.475696 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-766f66d7b8-t94xt" Jun 11 16:23:34.412688 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.412657 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-lkm68"] Jun 11 16:23:34.419388 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.419370 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:34.422692 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.422670 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6pb4r\"" Jun 11 16:23:34.422961 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.422923 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 11 16:23:34.426996 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.426972 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-lkm68"] Jun 11 16:23:34.525293 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.525261 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nvz6\" (UniqueName: \"kubernetes.io/projected/d3e4746f-e207-4191-a298-380d5431e400-kube-api-access-9nvz6\") pod \"payload-processing-699b6cd6c-lkm68\" (UID: \"d3e4746f-e207-4191-a298-380d5431e400\") " pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:34.585206 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.585178 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:23:34.626272 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.626244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nvz6\" (UniqueName: \"kubernetes.io/projected/d3e4746f-e207-4191-a298-380d5431e400-kube-api-access-9nvz6\") pod \"payload-processing-699b6cd6c-lkm68\" (UID: \"d3e4746f-e207-4191-a298-380d5431e400\") " pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:34.637827 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.637803 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nvz6\" (UniqueName: \"kubernetes.io/projected/d3e4746f-e207-4191-a298-380d5431e400-kube-api-access-9nvz6\") pod \"payload-processing-699b6cd6c-lkm68\" (UID: \"d3e4746f-e207-4191-a298-380d5431e400\") " pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:34.731361 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.731302 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:34.868077 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:34.868053 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-699b6cd6c-lkm68"] Jun 11 16:23:34.870431 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:23:34.870393 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e4746f_e207_4191_a298_380d5431e400.slice/crio-c40bebbc046821e24f40a91b4cda9a44a18276ec53dc306d0c9210b69da66318 WatchSource:0}: Error finding container c40bebbc046821e24f40a91b4cda9a44a18276ec53dc306d0c9210b69da66318: Status 404 returned error can't find the container with id c40bebbc046821e24f40a91b4cda9a44a18276ec53dc306d0c9210b69da66318 Jun 11 16:23:35.630315 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:35.630270 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" event={"ID":"d3e4746f-e207-4191-a298-380d5431e400","Type":"ContainerStarted","Data":"c40bebbc046821e24f40a91b4cda9a44a18276ec53dc306d0c9210b69da66318"} Jun 11 16:23:37.640237 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:37.640200 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" event={"ID":"d3e4746f-e207-4191-a298-380d5431e400","Type":"ContainerStarted","Data":"fea0b49dbfb81b6d1cc58f703b24e3758fdd43e302dddb369695c05731aa3c56"} Jun 11 16:23:37.640579 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:37.640331 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:37.665090 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:37.665046 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" podStartSLOduration=1.475208906 podStartE2EDuration="3.665031438s" totalImagesPullingTime="2.189822532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:23:37.662359252 +0000 UTC m=+882.027713425" watchObservedRunningTime="2026-06-11 16:23:37.665031438 +0000 UTC m=+882.030385610" Jun 11 16:23:48.647746 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:48.647664 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-699b6cd6c-lkm68" Jun 11 16:23:56.090511 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:56.090406 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:23:56.106991 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:23:56.091861 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:26:45.529585 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.529497 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf"] Jun 11 16:26:45.533393 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.533365 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.536091 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.536069 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 16:26:45.537103 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.537086 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-26ghs\"" Jun 11 16:26:45.537189 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.537108 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 11 16:26:45.537189 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.537160 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 16:26:45.544521 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.544500 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf"] Jun 11 16:26:45.586248 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.586222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.586377 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.586261 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p4v4\" (UniqueName: \"kubernetes.io/projected/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-kube-api-access-7p4v4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.586377 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.586341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.586377 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.586364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.586481 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.586390 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.586481 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.586420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687311 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687455 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p4v4\" (UniqueName: \"kubernetes.io/projected/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-kube-api-access-7p4v4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687455 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687455 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687455 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687437 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687603 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687839 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687815 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.687839 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687838 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.688152 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.687863 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.689586 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.689563 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.690018 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.690000 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.695754 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.695733 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p4v4\" (UniqueName: \"kubernetes.io/projected/7fb1fdcf-e916-4943-9fa6-6c82cf86d960-kube-api-access-7p4v4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf\" (UID: \"7fb1fdcf-e916-4943-9fa6-6c82cf86d960\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.843899 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.843833 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:26:45.982984 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.982951 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf"] Jun 11 16:26:45.984345 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:26:45.984302 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fb1fdcf_e916_4943_9fa6_6c82cf86d960.slice/crio-e14583289387860278b675419ea47e183ad3f3ef05ffe5577cd7ba8a10267f56 WatchSource:0}: Error finding container e14583289387860278b675419ea47e183ad3f3ef05ffe5577cd7ba8a10267f56: Status 404 returned error can't find the container with id e14583289387860278b675419ea47e183ad3f3ef05ffe5577cd7ba8a10267f56 Jun 11 16:26:45.986214 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.986199 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:26:45.993846 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:45.993823 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:26:46.539193 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:46.539155 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" event={"ID":"7fb1fdcf-e916-4943-9fa6-6c82cf86d960","Type":"ContainerStarted","Data":"e14583289387860278b675419ea47e183ad3f3ef05ffe5577cd7ba8a10267f56"} Jun 11 16:26:51.566295 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:51.566258 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" event={"ID":"7fb1fdcf-e916-4943-9fa6-6c82cf86d960","Type":"ContainerStarted","Data":"67b5c6630f1e202ecd4a893a5a8eda0bb1ab426cd0c8868d8d13237c959da941"} Jun 11 16:26:51.693618 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:26:51.693584 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:27:03.614029 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.613989 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw"] Jun 11 16:27:03.620780 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.620750 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.623436 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.623412 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 11 16:27:03.632528 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.632495 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw"] Jun 11 16:27:03.750862 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.750828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44m7q\" (UniqueName: \"kubernetes.io/projected/37287d93-a9aa-444b-8ee8-b4a5a26c4357-kube-api-access-44m7q\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.751055 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.750870 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.751055 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.750972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.751055 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.750992 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.751055 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.751010 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37287d93-a9aa-444b-8ee8-b4a5a26c4357-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.751055 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.751036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852294 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852262 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852448 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852448 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37287d93-a9aa-444b-8ee8-b4a5a26c4357-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852448 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852448 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852383 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44m7q\" (UniqueName: \"kubernetes.io/projected/37287d93-a9aa-444b-8ee8-b4a5a26c4357-kube-api-access-44m7q\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852448 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852776 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852742 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852776 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852773 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.852960 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.852830 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.854780 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.854760 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37287d93-a9aa-444b-8ee8-b4a5a26c4357-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.855112 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.855091 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37287d93-a9aa-444b-8ee8-b4a5a26c4357-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.861316 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.861298 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44m7q\" (UniqueName: \"kubernetes.io/projected/37287d93-a9aa-444b-8ee8-b4a5a26c4357-kube-api-access-44m7q\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw\" (UID: \"37287d93-a9aa-444b-8ee8-b4a5a26c4357\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:03.932332 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:03.932279 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:27:04.069778 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:04.069752 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw"] Jun 11 16:27:04.071717 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:27:04.071689 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37287d93_a9aa_444b_8ee8_b4a5a26c4357.slice/crio-d76b406540f05bf9d5af93f0b30a91d8b6bcd2672c319c7c0b19cdd6013177d2 WatchSource:0}: Error finding container d76b406540f05bf9d5af93f0b30a91d8b6bcd2672c319c7c0b19cdd6013177d2: Status 404 returned error can't find the container with id d76b406540f05bf9d5af93f0b30a91d8b6bcd2672c319c7c0b19cdd6013177d2 Jun 11 16:27:04.285406 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:04.285372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:27:04.630509 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:04.630431 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" event={"ID":"37287d93-a9aa-444b-8ee8-b4a5a26c4357","Type":"ContainerStarted","Data":"2915c6b282e5d2d6ac76130b6c8e1474f6b3643db199f2ae26bcc9368141398e"} Jun 11 16:27:04.630509 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:04.630486 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" event={"ID":"37287d93-a9aa-444b-8ee8-b4a5a26c4357","Type":"ContainerStarted","Data":"d76b406540f05bf9d5af93f0b30a91d8b6bcd2672c319c7c0b19cdd6013177d2"} Jun 11 16:27:13.193942 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:13.193906 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:27:18.278987 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:18.278954 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:27:24.642718 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.642684 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk"] Jun 11 16:27:24.646524 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.646504 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.650673 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.650651 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 11 16:27:24.662303 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.662282 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk"] Jun 11 16:27:24.751080 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.751054 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.751209 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.751089 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.751209 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.751137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5q7z\" (UniqueName: \"kubernetes.io/projected/31673586-5847-4b74-9227-8dd643cab951-kube-api-access-j5q7z\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.751283 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.751208 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31673586-5847-4b74-9227-8dd643cab951-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.751283 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.751256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.751361 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.751341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852273 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852316 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5q7z\" (UniqueName: \"kubernetes.io/projected/31673586-5847-4b74-9227-8dd643cab951-kube-api-access-j5q7z\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852351 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31673586-5847-4b74-9227-8dd643cab951-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852432 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852600 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852600 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852701 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852676 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.852800 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852775 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.853002 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.852973 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.854803 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.854779 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31673586-5847-4b74-9227-8dd643cab951-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.854910 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.854857 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31673586-5847-4b74-9227-8dd643cab951-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.862809 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.862784 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5q7z\" (UniqueName: \"kubernetes.io/projected/31673586-5847-4b74-9227-8dd643cab951-kube-api-access-j5q7z\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-n6tmk\" (UID: \"31673586-5847-4b74-9227-8dd643cab951\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:24.956542 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:24.956428 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" Jun 11 16:27:25.101992 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:25.101966 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk"] Jun 11 16:27:25.103212 ip-10-0-128-161 kubenswrapper[2560]: W0611 16:27:25.103192 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31673586_5847_4b74_9227_8dd643cab951.slice/crio-1dc630c13f23f47c8b908bdb90c76dc80502ddb037e5d339eef1b5280e22b359 WatchSource:0}: Error finding container 1dc630c13f23f47c8b908bdb90c76dc80502ddb037e5d339eef1b5280e22b359: Status 404 returned error can't find the container with id 1dc630c13f23f47c8b908bdb90c76dc80502ddb037e5d339eef1b5280e22b359 Jun 11 16:27:25.729461 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:25.729429 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" event={"ID":"31673586-5847-4b74-9227-8dd643cab951","Type":"ContainerStarted","Data":"147700942d57373ff39a475e6bee81443f52690c61d03272918160660f465b2f"} Jun 11 16:27:25.729797 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:25.729471 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-n6tmk" event={"ID":"31673586-5847-4b74-9227-8dd643cab951","Type":"ContainerStarted","Data":"1dc630c13f23f47c8b908bdb90c76dc80502ddb037e5d339eef1b5280e22b359"} Jun 11 16:27:25.788076 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:25.788050 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xhdft"] Jun 11 16:27:43.818883 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:43.818848 2560 generic.go:356] "Generic (PLEG): container finished" podID="7fb1fdcf-e916-4943-9fa6-6c82cf86d960" containerID="67b5c6630f1e202ecd4a893a5a8eda0bb1ab426cd0c8868d8d13237c959da941" exitCode=0 Jun 11 16:27:43.819312 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:43.818970 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" event={"ID":"7fb1fdcf-e916-4943-9fa6-6c82cf86d960","Type":"ContainerDied","Data":"67b5c6630f1e202ecd4a893a5a8eda0bb1ab426cd0c8868d8d13237c959da941"} Jun 11 16:27:45.830322 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:45.830283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" event={"ID":"7fb1fdcf-e916-4943-9fa6-6c82cf86d960","Type":"ContainerStarted","Data":"8e935f3adac02a83f55ab7909794ab2e487d3831d629c9f09f3aa5780ff22ea0"} Jun 11 16:27:45.844010 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:45.843979 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:27:45.844155 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:45.844039 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:27:45.853952 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:45.853887 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" podStartSLOduration=2.995855955 podStartE2EDuration="1m0.853869034s" totalImagesPullingTime="5.582873874s" totalInitContainerRuntime="52.275139205s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:26:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:27:45.853020028 +0000 UTC m=+1130.218374200" watchObservedRunningTime="2026-06-11 16:27:45.853869034 +0000 UTC m=+1130.219223207" Jun 11 16:27:45.856467 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:45.856444 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:27:46.846777 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:46.846747 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf" Jun 11 16:27:55.882026 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:55.881993 2560 generic.go:356] "Generic (PLEG): container finished" podID="37287d93-a9aa-444b-8ee8-b4a5a26c4357" containerID="2915c6b282e5d2d6ac76130b6c8e1474f6b3643db199f2ae26bcc9368141398e" exitCode=0 Jun 11 16:27:55.882418 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:55.882066 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" event={"ID":"37287d93-a9aa-444b-8ee8-b4a5a26c4357","Type":"ContainerDied","Data":"2915c6b282e5d2d6ac76130b6c8e1474f6b3643db199f2ae26bcc9368141398e"} Jun 11 16:27:56.888453 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:56.888416 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" event={"ID":"37287d93-a9aa-444b-8ee8-b4a5a26c4357","Type":"ContainerStarted","Data":"3ad87cfed39f04778869977bb2f3c3ed05a50193952d63c971a70d7ca8a0de14"} Jun 11 16:27:56.910059 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:27:56.909861 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" podStartSLOduration=2.2826998 podStartE2EDuration="53.909840729s" totalImagesPullingTime="304.640407ms" totalInitContainerRuntime="51.322500522s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:27:56.909021134 +0000 UTC m=+1141.274375317" watchObservedRunningTime="2026-06-11 16:27:56.909840729 +0000 UTC m=+1141.275194902" Jun 11 16:28:03.932882 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:28:03.932848 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:28:03.933472 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:28:03.932913 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:28:03.945149 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:28:03.945126 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:28:04.945308 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:28:04.945280 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw" Jun 11 16:28:56.122195 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:28:56.122092 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:28:56.128328 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:28:56.128305 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:33:19.758748 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:19.758719 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-n6tmk_31673586-5847-4b74-9227-8dd643cab951/storage-initializer/0.log" Jun 11 16:33:26.101733 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:26.101701 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-766f66d7b8-t94xt_103d0c3b-7aa8-4351-a0a8-044447e89cf4/manager/0.log" Jun 11 16:33:26.681317 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:26.681288 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-z558w_4963d659-4217-4cf0-8c8f-379d2724386b/copy-manifests/0.log" Jun 11 16:33:26.689186 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:26.687865 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-z558w_4963d659-4217-4cf0-8c8f-379d2724386b/manager/0.log" Jun 11 16:33:28.395922 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:28.395890 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gd54s_e360d1af-44e6-4dff-a1e4-2406d73392e7/kuadrant-console-plugin/0.log" Jun 11 16:33:28.746735 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:28.746707 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xhdft_63545728-15ed-4587-b1bf-c1448f8986cf/limitador/0.log" Jun 11 16:33:29.420590 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:29.420557 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-g96ms_ee854e50-564a-4b1d-8368-193fde514639/kube-auth-proxy/0.log" Jun 11 16:33:29.856674 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:29.856642 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-lkm68_d3e4746f-e207-4191-a298-380d5431e400/payload-processing/0.log" Jun 11 16:33:30.299294 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:30.299266 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf_7fb1fdcf-e916-4943-9fa6-6c82cf86d960/storage-initializer/0.log" Jun 11 16:33:30.306384 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:30.306361 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-5ksjf_7fb1fdcf-e916-4943-9fa6-6c82cf86d960/main/0.log" Jun 11 16:33:30.526109 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:30.526063 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw_37287d93-a9aa-444b-8ee8-b4a5a26c4357/storage-initializer/0.log" Jun 11 16:33:30.532621 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:30.532602 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-4j9nw_37287d93-a9aa-444b-8ee8-b4a5a26c4357/main/0.log" Jun 11 16:33:30.883725 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:30.883696 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-n6tmk_31673586-5847-4b74-9227-8dd643cab951/storage-initializer/0.log" Jun 11 16:33:37.846194 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:37.846162 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xbj5v_0ff4b19f-3067-44b8-bf80-4a7041a91297/global-pull-secret-syncer/0.log" Jun 11 16:33:37.921184 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:37.921156 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mvs4q_89d25600-3147-44e1-81e3-46f44c27e2ba/konnectivity-agent/0.log" Jun 11 16:33:37.976536 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:37.976500 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-161.ec2.internal_62b70f08a0210789f3a7a3206903da55/haproxy/0.log" Jun 11 16:33:42.821824 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:42.821789 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gd54s_e360d1af-44e6-4dff-a1e4-2406d73392e7/kuadrant-console-plugin/0.log" Jun 11 16:33:42.958478 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:42.958451 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xhdft_63545728-15ed-4587-b1bf-c1448f8986cf/limitador/0.log" Jun 11 16:33:44.605759 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.605728 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/alertmanager/0.log" Jun 11 16:33:44.638807 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.638778 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/config-reloader/0.log" Jun 11 16:33:44.661467 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.661447 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/kube-rbac-proxy-web/0.log" Jun 11 16:33:44.694800 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.694774 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/kube-rbac-proxy/0.log" Jun 11 16:33:44.716730 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.716709 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/kube-rbac-proxy-metric/0.log" Jun 11 16:33:44.740635 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.740609 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/prom-label-proxy/0.log" Jun 11 16:33:44.770802 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.770781 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d52c2df3-bb5d-4770-849a-99538ffbc888/init-config-reloader/0.log" Jun 11 16:33:44.814700 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:44.814676 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-74bbf69bbb-rfvb5_2b56d67a-a366-4ea7-be50-7f7f793cc700/cluster-monitoring-operator/0.log" Jun 11 16:33:45.013840 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.013790 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mn7j_3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f/node-exporter/0.log" Jun 11 16:33:45.037274 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.037252 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mn7j_3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f/kube-rbac-proxy/0.log" Jun 11 16:33:45.058722 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.058702 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mn7j_3d6bf72c-b5e1-47e1-83b6-bcea3bafdb9f/init-textfile/0.log" Jun 11 16:33:45.381287 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.381251 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/prometheus/0.log" Jun 11 16:33:45.403836 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.403810 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/config-reloader/0.log" Jun 11 16:33:45.425846 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.425827 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/thanos-sidecar/0.log" Jun 11 16:33:45.446518 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.446499 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/kube-rbac-proxy-web/0.log" Jun 11 16:33:45.468130 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.468109 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/kube-rbac-proxy/0.log" Jun 11 16:33:45.488693 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.488673 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/kube-rbac-proxy-thanos/0.log" Jun 11 16:33:45.510433 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.510417 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9d349060-7659-4578-a87c-f73ca77da27c/init-config-reloader/0.log" Jun 11 16:33:45.688690 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.688613 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9bbf5687c-5cfgj_8741f554-4e17-4c65-84e7-ac9221141bbc/telemeter-client/0.log" Jun 11 16:33:45.743332 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.743307 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9bbf5687c-5cfgj_8741f554-4e17-4c65-84e7-ac9221141bbc/reload/0.log" Jun 11 16:33:45.766769 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:45.766747 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9bbf5687c-5cfgj_8741f554-4e17-4c65-84e7-ac9221141bbc/kube-rbac-proxy/0.log" Jun 11 16:33:47.241765 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:47.241733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-fqlwz_209621f8-7603-4523-b0bb-d4e1f61b6913/networking-console-plugin/0.log" Jun 11 16:33:47.903831 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:47.903805 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:33:47.907528 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:47.907503 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/3.log" Jun 11 16:33:48.209712 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.209623 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf"] Jun 11 16:33:48.213455 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.213434 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.218126 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.218104 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvx9r\"/\"kube-root-ca.crt\"" Jun 11 16:33:48.219156 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.219136 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvx9r\"/\"openshift-service-ca.crt\"" Jun 11 16:33:48.219281 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.219174 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fvx9r\"/\"default-dockercfg-xmzhk\"" Jun 11 16:33:48.227356 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.227335 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf"] Jun 11 16:33:48.293712 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.293689 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-proc\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.294061 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.293740 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-lib-modules\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.294061 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.293807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-sys\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.294061 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.293856 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-podres\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.294061 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.293889 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcqmr\" (UniqueName: \"kubernetes.io/projected/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-kube-api-access-wcqmr\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394443 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-podres\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394528 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcqmr\" (UniqueName: \"kubernetes.io/projected/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-kube-api-access-wcqmr\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394528 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-proc\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394595 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-lib-modules\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394595 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-sys\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394595 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394575 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-podres\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394708 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394624 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-proc\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394708 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-sys\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.394708 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.394682 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-lib-modules\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.404042 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.404024 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcqmr\" (UniqueName: \"kubernetes.io/projected/eac5d8c2-5529-4a84-8d2a-ebf85deb3c60-kube-api-access-wcqmr\") pod \"perf-node-gather-daemonset-trsdf\" (UID: \"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.523495 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.523455 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:48.665963 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:48.661234 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf"] Jun 11 16:33:49.292011 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:49.291982 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-rs2xr_c9c0c5be-ef6e-4864-b60f-889eff72ad24/volume-data-source-validator/0.log" Jun 11 16:33:49.624332 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:49.624242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" event={"ID":"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60","Type":"ContainerStarted","Data":"14858d3151712d016421c5bcaae9515d4e51dfeb82d5811e7cfdd05183e392d7"} Jun 11 16:33:49.624332 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:49.624303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" event={"ID":"eac5d8c2-5529-4a84-8d2a-ebf85deb3c60","Type":"ContainerStarted","Data":"a61989cae7ca0f720286561ab7b83bd1351e1dedceb317247dbefc4d66f56229"} Jun 11 16:33:49.624710 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:49.624358 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:49.649443 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:49.649389 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" podStartSLOduration=1.649374117 podStartE2EDuration="1.649374117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:33:49.647437331 +0000 UTC m=+1494.012791504" watchObservedRunningTime="2026-06-11 16:33:49.649374117 +0000 UTC m=+1494.014728289" Jun 11 16:33:50.587578 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:50.587556 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z2szh_09604985-6b11-49bc-98d6-1331719d4f16/dns/0.log" Jun 11 16:33:50.616143 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:50.616110 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-z2szh_09604985-6b11-49bc-98d6-1331719d4f16/kube-rbac-proxy/0.log" Jun 11 16:33:50.640392 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:50.640370 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-275rc_ad8f72cc-af12-44c9-9715-c07563601aee/dns-node-resolver/0.log" Jun 11 16:33:51.274835 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:51.274808 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-658f6f668-s6lng_0101db04-d384-4db3-a0b8-d17758cf3c00/registry/0.log" Jun 11 16:33:51.356296 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:51.356270 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pcqw6_03b1995d-2d6e-46d6-9ed7-5bb31b6dbadf/node-ca/0.log" Jun 11 16:33:52.451028 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:52.450999 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-g96ms_ee854e50-564a-4b1d-8368-193fde514639/kube-auth-proxy/0.log" Jun 11 16:33:52.553606 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:52.553582 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-lkm68_d3e4746f-e207-4191-a298-380d5431e400/payload-processing/0.log" Jun 11 16:33:53.229410 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:53.229374 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7bmts_73dcc6de-fa1c-4145-a113-653c0863671e/serve-healthcheck-canary/0.log" Jun 11 16:33:53.804831 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:53.804804 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-z5kgd_559452f3-f51f-44c6-8997-04f31fcae598/insights-operator/0.log" Jun 11 16:33:53.805238 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:53.804886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-z5kgd_559452f3-f51f-44c6-8997-04f31fcae598/insights-operator/1.log" Jun 11 16:33:53.828163 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:53.828136 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4kmvj_7d83da3d-65b6-496c-a9c5-c2efd392748e/kube-rbac-proxy/0.log" Jun 11 16:33:53.851491 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:53.851469 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4kmvj_7d83da3d-65b6-496c-a9c5-c2efd392748e/exporter/0.log" Jun 11 16:33:53.873752 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:53.873731 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4kmvj_7d83da3d-65b6-496c-a9c5-c2efd392748e/extractor/0.log" Jun 11 16:33:55.639359 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:55.639329 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-trsdf" Jun 11 16:33:56.154085 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:56.153969 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:33:56.160949 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:56.160903 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-jfsm9_9a3095c8-d355-4890-b985-846e3bb2e3e4/console-operator/2.log" Jun 11 16:33:56.639770 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:56.639740 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-766f66d7b8-t94xt_103d0c3b-7aa8-4351-a0a8-044447e89cf4/manager/0.log" Jun 11 16:33:56.948699 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:56.948632 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-z558w_4963d659-4217-4cf0-8c8f-379d2724386b/manager/0.log" Jun 11 16:33:56.972507 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:56.972474 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-z558w_4963d659-4217-4cf0-8c8f-379d2724386b/copy-manifests/0.log" Jun 11 16:33:58.878031 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:58.877998 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-67c4b4c7b5-kh9mx_abee4d4e-9369-422a-a771-cc5f1a567541/manager/0.log" Jun 11 16:33:58.919945 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:33:58.919907 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mhvn2_36be49fb-ffac-4080-9c77-633f4ac8fd58/openshift-lws-operator/0.log" Jun 11 16:34:02.916617 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:34:02.916588 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-g96ms_ee854e50-564a-4b1d-8368-193fde514639/kube-auth-proxy/0.log" Jun 11 16:34:02.978422 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:34:02.978394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-lkm68_d3e4746f-e207-4191-a298-380d5431e400/payload-processing/0.log" Jun 11 16:34:04.203260 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:34:04.203180 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-g96ms_ee854e50-564a-4b1d-8368-193fde514639/kube-auth-proxy/0.log" Jun 11 16:34:04.269974 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:34:04.269919 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-699b6cd6c-lkm68_d3e4746f-e207-4191-a298-380d5431e400/payload-processing/0.log" Jun 11 16:34:05.005602 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:34:05.005567 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-xv5fk_79be130d-d4cc-46d4-a56c-5e216839fcc5/kube-storage-version-migrator-operator/1.log" Jun 11 16:34:05.006353 ip-10-0-128-161 kubenswrapper[2560]: I0611 16:34:05.006335 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-xv5fk_79be130d-d4cc-46d4-a56c-5e216839fcc5/kube-storage-version-migrator-operator/0.log"