Jul 23 21:49:33.780493 ip-10-0-139-32 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 21:49:33.780506 ip-10-0-139-32 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 21:49:33.780513 ip-10-0-139-32 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 21:49:33.780917 ip-10-0-139-32 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 21:49:42.923168 ip-10-0-139-32 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 21:49:42.923183 ip-10-0-139-32 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 670c8364752d49bfb956330c2c9f1cb0 -- Jul 23 21:51:25.993915 ip-10-0-139-32 systemd[1]: Starting Kubernetes Kubelet... Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.363926 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369069 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369082 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369087 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369091 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369094 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369097 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369100 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:26.508003 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369103 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:26.439503 ip-10-0-139-32 systemd[1]: Started Kubernetes Kubelet. Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369105 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369108 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369111 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369113 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369116 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369118 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369123 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369127 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369129 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369132 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369135 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369139 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369142 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369145 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369148 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369151 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369154 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369157 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369159 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:26.509847 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369162 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369165 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369169 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369171 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369174 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369176 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369179 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369181 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369184 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369187 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369189 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369192 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369194 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369197 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369199 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369202 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369205 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369207 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369210 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369213 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:26.511133 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369215 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369217 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369221 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369224 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369227 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369230 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369233 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369236 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369239 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369242 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369245 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369248 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369251 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369254 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369256 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369259 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369261 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369263 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369266 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369269 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:26.512196 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369271 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369274 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369278 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369280 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369284 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369287 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369290 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369292 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369295 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369297 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369300 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369302 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369305 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369307 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369310 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369313 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369316 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369318 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369322 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:26.513253 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369325 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369329 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369331 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369334 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369337 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369340 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369343 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369346 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369348 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369351 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369353 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369357 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.369359 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370087 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370093 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370096 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370099 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370102 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370106 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:26.514327 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370108 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370111 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370113 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370116 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370119 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370122 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370125 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370129 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370133 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370136 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370139 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370142 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370145 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370148 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370151 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370154 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370156 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370159 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370162 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:26.515191 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370165 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370168 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370170 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370173 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370175 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370178 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370181 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370184 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370187 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370190 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370192 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370196 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370198 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370202 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370205 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370207 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370210 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370213 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370215 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370218 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:26.516249 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370220 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370223 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370225 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370228 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370230 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370232 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370235 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370237 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370246 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370257 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370261 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370263 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370266 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370269 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370271 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370274 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370276 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370279 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370281 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370284 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:26.517128 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370287 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370289 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370292 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370294 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370297 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370299 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370304 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370308 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370310 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370313 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370315 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370318 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370320 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370323 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370325 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370328 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370330 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370333 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370337 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:26.518203 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370340 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370344 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370347 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370350 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370352 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370355 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370358 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370360 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370363 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370365 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370368 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370370 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370373 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370375 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370378 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370769 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370774 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370778 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370780 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370784 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:26.519064 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370787 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370789 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370792 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370795 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370797 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370800 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370802 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370805 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370807 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370810 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370812 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370815 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370818 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370823 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370826 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370829 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370832 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370835 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370841 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:26.520117 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370844 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370847 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370849 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370852 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370854 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370856 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370859 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370862 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370864 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370867 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370869 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370872 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370875 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370879 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370881 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370884 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370887 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370889 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370892 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:26.520978 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370894 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370896 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370900 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370903 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370905 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370908 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370910 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370914 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370916 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370919 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370922 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370924 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370928 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370931 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370933 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370936 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370938 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370941 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370944 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370946 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:26.521846 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370949 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370954 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370957 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370960 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370963 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370966 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370969 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370972 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370975 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370979 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370982 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370985 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370988 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370990 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370993 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370996 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.370999 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371001 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371004 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:26.522924 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371008 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371010 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371013 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371015 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371017 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371021 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371024 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371026 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371029 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371031 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371034 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371037 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371039 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371041 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371044 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371046 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371049 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371115 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371122 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371128 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371132 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 21:51:26.523797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371137 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371140 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371145 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371149 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371152 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371155 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371159 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371162 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371165 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371168 2579 flags.go:64] FLAG: --cgroup-root="" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371171 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371175 2579 flags.go:64] FLAG: --client-ca-file="" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371178 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371181 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371186 2579 flags.go:64] FLAG: --cluster-domain="" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371189 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371193 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371196 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371200 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371203 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371207 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371210 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371213 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371216 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 23 21:51:26.524881 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371219 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371222 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371225 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371228 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371232 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371235 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371238 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371240 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371244 2579 flags.go:64] FLAG: --enable-server="true" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371247 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371251 2579 flags.go:64] FLAG: --event-burst="100" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371254 2579 flags.go:64] FLAG: --event-qps="50" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371257 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371260 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371263 2579 flags.go:64] FLAG: --eviction-hard="" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371267 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371270 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371272 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371275 2579 flags.go:64] FLAG: --eviction-soft="" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371281 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371284 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371287 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371290 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371293 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371297 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 21:51:26.525947 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371300 2579 flags.go:64] FLAG: --feature-gates="" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371304 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371307 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371310 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371313 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371317 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371320 2579 flags.go:64] FLAG: --help="false" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371323 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-139-32.ec2.internal" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371326 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371329 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371332 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371335 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371339 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371342 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371345 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371347 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371350 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371354 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371357 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371360 2579 flags.go:64] FLAG: --kube-reserved="" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371363 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371366 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371369 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371372 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 21:51:26.527044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371375 2579 flags.go:64] FLAG: --lock-file="" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371378 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371382 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371385 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371391 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371394 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371396 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371400 2579 flags.go:64] FLAG: --logging-format="text" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371403 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371407 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371409 2579 flags.go:64] FLAG: --manifest-url="" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371412 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371416 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371420 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371424 2579 flags.go:64] FLAG: --max-pods="110" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371426 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371429 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371432 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371435 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371438 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371441 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371444 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371451 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371454 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371457 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 21:51:26.528114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371460 2579 flags.go:64] FLAG: --pod-cidr="" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371473 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371479 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371482 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371485 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371488 2579 flags.go:64] FLAG: --port="10250" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371491 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371494 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04c6e6cb235011a46" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371497 2579 flags.go:64] FLAG: --qos-reserved="" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371501 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371504 2579 flags.go:64] FLAG: --register-node="true" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371507 2579 flags.go:64] FLAG: --register-with-taints="" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371515 2579 flags.go:64] FLAG: --registry-burst="10" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371518 2579 flags.go:64] FLAG: --registry-qps="5" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371521 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371526 2579 flags.go:64] FLAG: --reserved-memory="" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371530 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371533 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371536 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371539 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371542 2579 flags.go:64] FLAG: --runonce="false" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371544 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371548 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371550 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371553 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 21:51:26.533426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371556 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371559 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371562 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371565 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371568 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371571 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371573 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371576 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371579 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371583 2579 flags.go:64] FLAG: --system-cgroups="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371586 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371604 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371607 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371610 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371615 2579 flags.go:64] FLAG: --tls-min-version="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371618 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371622 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371625 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371628 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371631 2579 flags.go:64] FLAG: --v="2" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371635 2579 flags.go:64] FLAG: --version="false" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371639 2579 flags.go:64] FLAG: --vmodule="" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371644 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.371647 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371761 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:26.557150 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371765 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371768 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371771 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371774 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371777 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371780 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371783 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371785 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371788 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371790 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371793 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371795 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371798 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371800 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371803 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371805 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371808 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371811 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371814 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371817 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:27.915019 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371819 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371821 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371824 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371829 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371831 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371834 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371837 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371839 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371842 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371847 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371850 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371854 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371857 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371860 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371863 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371866 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371868 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371871 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371874 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:27.916517 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371876 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371879 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371881 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371885 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371888 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371891 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371894 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371896 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371899 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371902 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371904 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371907 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371910 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371912 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371915 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371917 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371921 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371924 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371926 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:27.917795 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371929 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371931 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371934 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371937 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371940 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371942 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371945 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371947 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371950 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371953 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371955 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371958 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371960 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371963 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371965 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371968 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371970 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371973 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371976 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371979 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:27.919074 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371981 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371984 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371986 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371989 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371992 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371994 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371997 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.371999 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372002 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372005 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372008 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372010 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372013 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372015 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372018 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372021 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372024 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372028 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372031 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:27.919965 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.372034 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.372039 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.379897 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.379913 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.380583 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.380604 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380664 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380669 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380672 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380676 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380680 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380684 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380687 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380690 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:27.920914 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380693 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380695 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380699 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380701 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380704 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380707 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380710 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380712 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380715 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380718 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380721 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380724 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380726 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380729 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380731 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380734 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380737 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380739 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380742 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380745 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:27.921796 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380747 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380751 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380754 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380756 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380759 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380761 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380764 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380766 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380769 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380772 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380774 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380777 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380779 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380782 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380784 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380787 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380789 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380792 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380794 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380797 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:27.922732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380799 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380803 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380806 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380808 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380811 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380813 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380816 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380818 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380821 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380823 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380826 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380828 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380831 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380834 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380837 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380840 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380843 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380845 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380848 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380850 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:27.923671 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380853 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380856 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380858 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380861 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380863 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380866 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380869 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380872 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380875 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380879 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380882 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380889 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380893 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380897 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380900 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380903 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380906 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380909 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380912 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:27.924610 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380915 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380921 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380924 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380927 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380930 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380933 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380936 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380939 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380942 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380945 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380948 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.380950 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.380956 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381069 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381074 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:27.925534 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381077 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381080 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381083 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381086 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381089 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381091 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381094 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381096 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381099 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381106 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381109 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381111 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381114 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381117 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381119 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381122 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381124 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381127 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381130 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381134 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:27.926433 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381136 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381140 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381143 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381147 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381150 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381153 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381156 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381159 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381162 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381165 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381167 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381170 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381173 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381177 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381180 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381183 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381186 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381189 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381191 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:27.927360 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381194 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381197 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381201 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381203 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381206 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381209 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381211 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381214 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381217 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381219 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381222 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381224 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381228 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381231 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381234 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381236 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381239 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381242 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381244 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381247 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:27.928303 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381250 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381253 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381255 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381258 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381261 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381263 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381266 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381268 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381271 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381274 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381276 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381279 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381281 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381284 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381288 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381290 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381293 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381295 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381298 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381300 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:27.929239 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381303 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381305 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381308 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381311 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381315 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381318 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381320 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381323 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381325 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381328 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381331 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381334 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381337 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381340 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381343 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381346 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381348 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:27.930167 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:26.381351 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.381356 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.381477 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.383443 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.384741 2579 server.go:1019] "Starting client certificate rotation" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.384844 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.385450 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.387163 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.392845 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.402312 2579 log.go:25] "Validated CRI v1 runtime API" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.404294 2579 log.go:25] "Validated CRI v1 image API" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.405147 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 21:51:27.931079 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.408947 2579 fs.go:135] Filesystem UUIDs: map[6e0c3564-ff5e-4ff2-aadd-fae100fba78f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d5461f9a-05a9-498c-af73-fe056a28b002:/dev/nvme0n1p3] Jul 23 21:51:27.931949 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.408965 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.413627 2579 manager.go:216] Machine: {Timestamp:2026-07-23 21:51:26.412408515 +0000 UTC m=+0.293938207 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099941 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f099891765dd7041ca0707dfead5f SystemUUID:ec2f0998-9176-5dd7-041c-a0707dfead5f BootID:670c8364-752d-49bf-b956-330c2c9f1cb0 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3f:6e:c9:93:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3f:6e:c9:93:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:a5:3f:d6:3a:c3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.413738 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.413859 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.416987 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.417023 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-32.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.418237 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.418257 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 21:51:27.932353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.418268 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.418870 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.420103 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.425319 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.425944 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426078 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426106 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426129 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426144 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426157 2579 kubelet.go:398] "Adding apiserver pod source" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426169 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426580 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.426615 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.429198 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.431278 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.431306 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432245 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432260 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432265 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432270 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432280 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432286 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432294 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432299 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432305 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432311 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432324 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.432332 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.434433 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.434456 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.436527 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-32.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.437090 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.437090 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-32.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.438618 2579 server.go:1268] "Started kubelet" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.438847 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.438891 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.439080 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.440152 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.440454 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 21:51:27.932840 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.443712 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.443068 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-32.ec2.internal.18c509b1c4d71e52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-32.ec2.internal,UID:ip-10-0-139-32.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-32.ec2.internal,},FirstTimestamp:2026-07-23 21:51:26.43856341 +0000 UTC m=+0.320093143,LastTimestamp:2026-07-23 21:51:26.43856341 +0000 UTC m=+0.320093143,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-32.ec2.internal,}" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.447430 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.447432 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.447514 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.447991 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.447998 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448016 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448116 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448125 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448285 2579 factory.go:55] Registering systemd factory Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448299 2579 factory.go:223] Registration of the systemd container factory successfully Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448514 2579 factory.go:153] Registering CRI-O factory Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448528 2579 factory.go:223] Registration of the crio container factory successfully Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448571 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448609 2579 factory.go:103] Registering Raw factory Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.448625 2579 manager.go:1196] Started watching for new ooms in manager Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.449013 2579 manager.go:349] Starting recovery of all containers Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.459529 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-32.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.459623 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.464012 2579 manager.go:354] Recovery completed Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.468966 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.472902 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.472945 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.472957 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.473338 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j9jcc" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.473586 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.473613 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.473618 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.473633 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.474332 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.474356 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:27.938196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.474385 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.475074 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-32.ec2.internal.18c509b1c6e37f28 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-32.ec2.internal,UID:ip-10-0-139-32.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-32.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-32.ec2.internal,},FirstTimestamp:2026-07-23 21:51:26.472929064 +0000 UTC m=+0.354458759,LastTimestamp:2026-07-23 21:51:26.472929064 +0000 UTC m=+0.354458759,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-32.ec2.internal,}" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.475833 2579 policy_none.go:49] "None policy: Start" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.475846 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.475857 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.477862 2579 policy_none.go:47] "Start" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.480812 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j9jcc" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.483872 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-32.ec2.internal.18c509b1c6e3ce98 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-32.ec2.internal,UID:ip-10-0-139-32.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-32.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-32.ec2.internal,},FirstTimestamp:2026-07-23 21:51:26.4729494 +0000 UTC m=+0.354479092,LastTimestamp:2026-07-23 21:51:26.4729494 +0000 UTC m=+0.354479092,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-32.ec2.internal,}" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.515457 2579 manager.go:337] "Starting Device Plugin manager" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.515756 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.515773 2579 server.go:87] "Starting device plugin registration server" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.516047 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.516060 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.516189 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.516272 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.516281 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.516981 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.517043 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.548257 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.549458 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.549492 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.549518 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.549555 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.552094 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.617025 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.618124 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.618150 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.618160 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.036099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.618186 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.627660 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.627680 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-32.ec2.internal\": node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.650208 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal"] Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.650278 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.651081 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.651106 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.651115 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653170 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653303 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653811 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653811 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653863 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653874 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653840 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.653910 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.655892 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.655920 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.656336 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.656544 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.656567 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.656580 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.684363 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-32.ec2.internal\" not found" node="ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.688818 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-32.ec2.internal\" not found" node="ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.749470 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/78661e095b3e65c966e2cbaf1488640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal\" (UID: \"78661e095b3e65c966e2cbaf1488640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.749496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78661e095b3e65c966e2cbaf1488640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal\" (UID: \"78661e095b3e65c966e2cbaf1488640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.037700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:26.749513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/634343fa5cba8d6321e15d7ff451094e-config\") pod \"kube-apiserver-proxy-ip-10-0-139-32.ec2.internal\" (UID: \"634343fa5cba8d6321e15d7ff451094e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:26.756576 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:27.972498 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/78661e095b3e65c966e2cbaf1488640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal\" (UID: \"78661e095b3e65c966e2cbaf1488640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:27.972534 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78661e095b3e65c966e2cbaf1488640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal\" (UID: \"78661e095b3e65c966e2cbaf1488640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:27.972551 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/634343fa5cba8d6321e15d7ff451094e-config\") pod \"kube-apiserver-proxy-ip-10-0-139-32.ec2.internal\" (UID: \"634343fa5cba8d6321e15d7ff451094e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:27.972590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78661e095b3e65c966e2cbaf1488640c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal\" (UID: \"78661e095b3e65c966e2cbaf1488640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:27.972589 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/78661e095b3e65c966e2cbaf1488640c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal\" (UID: \"78661e095b3e65c966e2cbaf1488640c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:27.972641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/634343fa5cba8d6321e15d7ff451094e-config\") pod \"kube-apiserver-proxy-ip-10-0-139-32.ec2.internal\" (UID: \"634343fa5cba8d6321e15d7ff451094e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.038490 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:27.979588 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.141397 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.079773 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.141397 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.109943 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.141397 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.114104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" Jul 23 21:51:28.243035 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.180595 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.380264 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.281589 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.382742 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.382063 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.483397 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.482504 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.584307 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.506652 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 21:51:28.584307 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.506797 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 21:51:28.584307 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.582736 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.688062 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.607707 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 21:46:27 +0000 UTC" deadline="2027-12-20 06:19:49.378310709 +0000 UTC" Jul 23 21:51:28.688062 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.607742 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12344h28m20.770572248s" Jul 23 21:51:28.688062 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.683096 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.699579 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.698254 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:28.699579 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.699246 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:28.699579 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.699278 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:28.699579 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.699288 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:28.797928 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.784019 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.817803 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.816853 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:28.904632 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.884825 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:28.915290 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:28.912754 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:29.018813 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:28.985613 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:29.127812 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.086226 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-32.ec2.internal\" not found" Jul 23 21:51:29.244681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.159242 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:29.244681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.171039 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" Jul 23 21:51:29.244681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.190049 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 21:51:29.244681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.191046 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" Jul 23 21:51:29.244681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.201030 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 21:51:29.458116 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.458079 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78661e095b3e65c966e2cbaf1488640c.slice/crio-f10f980b68e94e6f4cb9a29080a8d6dda77b76a9f4926b7b9a4104e6c1b72818 WatchSource:0}: Error finding container f10f980b68e94e6f4cb9a29080a8d6dda77b76a9f4926b7b9a4104e6c1b72818: Status 404 returned error can't find the container with id f10f980b68e94e6f4cb9a29080a8d6dda77b76a9f4926b7b9a4104e6c1b72818 Jul 23 21:51:29.458376 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.458361 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634343fa5cba8d6321e15d7ff451094e.slice/crio-bded84fca5ce4ac98fa11fa3f2c4d2c43d17a874768134ac3266e5acd5918ab0 WatchSource:0}: Error finding container bded84fca5ce4ac98fa11fa3f2c4d2c43d17a874768134ac3266e5acd5918ab0: Status 404 returned error can't find the container with id bded84fca5ce4ac98fa11fa3f2c4d2c43d17a874768134ac3266e5acd5918ab0 Jul 23 21:51:29.462697 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.462680 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 21:51:29.542446 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.542411 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:29.549618 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.549597 2579 apiserver.go:52] "Watching apiserver" Jul 23 21:51:29.560985 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.560953 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 21:51:29.561326 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.561306 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-mrck5","openshift-image-registry/node-ca-w4lz2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal","openshift-network-diagnostics/network-check-target-kncqt","openshift-network-operator/iptables-alerter-f94zf","openshift-ovn-kubernetes/ovnkube-node-rlb2q","kube-system/global-pull-secret-syncer-mgtn9","kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx","openshift-cluster-node-tuning-operator/tuned-g6l4d","openshift-dns/node-resolver-rqrtl","openshift-multus/multus-additional-cni-plugins-rlmqv","openshift-multus/multus-ntkwx","openshift-multus/network-metrics-daemon-gp9l9"] Jul 23 21:51:29.565103 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.565082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.567195 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.567177 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.568102 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.568088 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 21:51:29.568188 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.568103 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mg6mh\"" Jul 23 21:51:29.568252 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.568205 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 21:51:29.569338 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.569324 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.570173 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.570156 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 21:51:29.570417 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.570401 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.570620 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.570490 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.571600 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.571408 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 21:51:29.571600 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.571504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:29.571600 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.571563 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:29.572259 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.572241 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8dfmj\"" Jul 23 21:51:29.572324 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.572304 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 21:51:29.572450 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.572434 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-27b97\"" Jul 23 21:51:29.573619 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.573594 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 21:51:29.573714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.573660 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.573714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.573677 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.573825 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.573773 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.575883 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.575869 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.577990 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.577975 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.578086 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.578041 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:29.578383 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.578369 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.578460 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.578443 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.578571 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.578557 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ds2rt\"" Jul 23 21:51:29.578643 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.578628 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8zfks\"" Jul 23 21:51:29.578689 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.578627 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 21:51:29.579424 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.579405 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.579508 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.579406 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.580349 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.580332 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.581429 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581414 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11144f63-f9a8-4fcc-a309-56aee07f6423-cni-binary-copy\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581524 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581435 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-conf-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581524 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-daemon-config\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581524 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581467 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-multus-certs\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581524 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581488 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-host\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.581524 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581511 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh7tn\" (UniqueName: \"kubernetes.io/projected/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-kube-api-access-rh7tn\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581528 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-hostroot\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581574 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-serviceca\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581598 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-system-cni-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581614 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-cni-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581628 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-os-release\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581652 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-socket-dir-parent\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581673 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-netns\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581690 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-k8s-cni-cncf-io\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581720 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-cni-bin\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581735 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-cni-multus\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581749 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-kubelet\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.581760 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581762 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-etc-kubernetes\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.582321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581804 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhndm\" (UniqueName: \"kubernetes.io/projected/11144f63-f9a8-4fcc-a309-56aee07f6423-kube-api-access-zhndm\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.582321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581825 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:29.582321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581843 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f35f40c2-adf4-4b5c-831f-c9b9f0c40b09-agent-certs\") pod \"konnectivity-agent-mrck5\" (UID: \"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09\") " pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.582321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581860 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f35f40c2-adf4-4b5c-831f-c9b9f0c40b09-konnectivity-ca\") pod \"konnectivity-agent-mrck5\" (UID: \"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09\") " pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.582321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.581886 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-cnibin\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.585202 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.585154 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.585359 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.585343 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gf6qx\"" Jul 23 21:51:29.585436 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.585213 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 21:51:29.585567 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.585488 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.586768 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.586752 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.587938 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.587924 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.588295 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.588278 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.588568 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.588549 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c8tnz\"" Jul 23 21:51:29.588677 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.588656 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.591015 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.591001 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.593202 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.593173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:29.593294 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.593222 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:29.595381 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.595363 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 21:51:29.595623 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.595606 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 21:51:29.596914 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.596899 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 21:51:29.596978 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.596919 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4nl8p\"" Jul 23 21:51:29.597015 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.596975 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 21:51:29.597015 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.597008 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 21:51:29.597168 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.597153 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 21:51:29.600675 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.600658 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 21:51:29.601214 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.601199 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 21:51:29.601781 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.601768 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6g78c\"" Jul 23 21:51:29.603185 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.603167 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 21:51:29.626736 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.626713 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tc58d" Jul 23 21:51:29.637205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.637185 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tc58d" Jul 23 21:51:29.671805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.671777 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 21:51:29.675747 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.675708 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" event={"ID":"78661e095b3e65c966e2cbaf1488640c","Type":"ContainerStarted","Data":"f10f980b68e94e6f4cb9a29080a8d6dda77b76a9f4926b7b9a4104e6c1b72818"} Jul 23 21:51:29.676631 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.676612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" event={"ID":"634343fa5cba8d6321e15d7ff451094e","Type":"ContainerStarted","Data":"bded84fca5ce4ac98fa11fa3f2c4d2c43d17a874768134ac3266e5acd5918ab0"} Jul 23 21:51:29.682854 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682833 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-serviceca\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.682919 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682863 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-ovn\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.682919 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682881 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-kubelet\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.682919 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682895 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-etc-kubernetes\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.682919 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682911 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-host-slash\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-var-lib-kubelet\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682968 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e2f2115-998c-4843-bf33-1f7758ba633e-tmp\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682975 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-etc-kubernetes\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.682989 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-cnibin\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683021 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-kubelet\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683036 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11144f63-f9a8-4fcc-a309-56aee07f6423-cni-binary-copy\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683045 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-cnibin\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683092 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-multus-certs\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-var-lib-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkk5c\" (UniqueName: \"kubernetes.io/projected/1914f60a-ab07-4a57-8c3e-05cdcc59407c-kube-api-access-gkk5c\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683184 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af88a0e-a538-435c-a540-65e938e7ecf6-tmp-dir\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683223 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-multus-certs\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-host\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683263 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-etc-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683286 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-log-socket\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683320 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683347 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-cni-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovnkube-config\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683392 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovn-node-metrics-cert\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4lgf\" (UniqueName: \"kubernetes.io/projected/21a14840-5dfc-4c8d-91b1-ef5db29807f6-kube-api-access-f4lgf\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683445 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683485 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-serviceca\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683494 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-run-netns\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-env-overrides\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683544 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-cni-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-conf-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683581 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-conf-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683588 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-daemon-config\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-socket-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683631 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-node-log\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683672 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11144f63-f9a8-4fcc-a309-56aee07f6423-cni-binary-copy\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683737 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-system-cni-dir\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683788 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-sys\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683840 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-host\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.683994 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.683879 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-hostroot\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684009 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-host\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw2st\" (UniqueName: \"kubernetes.io/projected/5af88a0e-a538-435c-a540-65e938e7ecf6-kube-api-access-gw2st\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684049 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-hostroot\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-dbus\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684142 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-device-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684166 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw7r6\" (UniqueName: \"kubernetes.io/projected/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-kube-api-access-bw7r6\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-systemd\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684214 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-daemon-config\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684228 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-system-cni-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684267 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-os-release\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684288 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-socket-dir-parent\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684346 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-iptables-alerter-script\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684365 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-system-cni-dir\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.684491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684376 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-os-release\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysctl-d\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684387 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-multus-socket-dir-parent\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684414 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-slash\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684430 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-registration-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovnkube-script-lib\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684537 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-modprobe-d\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684575 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684591 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-cni-bin\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh7tn\" (UniqueName: \"kubernetes.io/projected/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-kube-api-access-rh7tn\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684650 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.685041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvsgl\" (UniqueName: \"kubernetes.io/projected/8e2f2115-998c-4843-bf33-1f7758ba633e-kube-api-access-zvsgl\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-systemd-units\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cnibin\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684769 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-netns\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684855 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhndm\" (UniqueName: \"kubernetes.io/projected/11144f63-f9a8-4fcc-a309-56aee07f6423-kube-api-access-zhndm\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684876 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-kubernetes\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684900 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-netns\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684890 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-run\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-cni-netd\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684982 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxr2n\" (UniqueName: \"kubernetes.io/projected/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-kube-api-access-zxr2n\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.684998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-lib-modules\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685013 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-tuned\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.685612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685049 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-k8s-cni-cncf-io\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-cni-bin\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685120 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-run-k8s-cni-cncf-io\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-cni-multus\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-kubelet-config\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685219 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-cni-multus\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685232 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2rnb\" (UniqueName: \"kubernetes.io/projected/372108ce-3670-4e52-9da3-6d8a0df670eb-kube-api-access-c2rnb\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685245 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-systemd\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11144f63-f9a8-4fcc-a309-56aee07f6423-host-var-lib-cni-bin\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f35f40c2-adf4-4b5c-831f-c9b9f0c40b09-agent-certs\") pod \"konnectivity-agent-mrck5\" (UID: \"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09\") " pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685324 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f35f40c2-adf4-4b5c-831f-c9b9f0c40b09-konnectivity-ca\") pod \"konnectivity-agent-mrck5\" (UID: \"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09\") " pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685370 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5af88a0e-a538-435c-a540-65e938e7ecf6-hosts-file\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685400 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:29.686009 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685424 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysconfig\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.686416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685486 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-os-release\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.686416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685516 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysctl-conf\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.686416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685549 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-sys-fs\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.686416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-kubelet\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.686416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685593 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 21:51:29.686416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.685798 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f35f40c2-adf4-4b5c-831f-c9b9f0c40b09-konnectivity-ca\") pod \"konnectivity-agent-mrck5\" (UID: \"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09\") " pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.688859 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.688842 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f35f40c2-adf4-4b5c-831f-c9b9f0c40b09-agent-certs\") pod \"konnectivity-agent-mrck5\" (UID: \"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09\") " pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.693116 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.693099 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:29.693197 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.693120 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:29.693197 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.693130 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:29.693275 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.693209 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:30.193175801 +0000 UTC m=+2.952246808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:29.694807 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.694791 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhndm\" (UniqueName: \"kubernetes.io/projected/11144f63-f9a8-4fcc-a309-56aee07f6423-kube-api-access-zhndm\") pod \"multus-ntkwx\" (UID: \"11144f63-f9a8-4fcc-a309-56aee07f6423\") " pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.695005 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.694989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh7tn\" (UniqueName: \"kubernetes.io/projected/7c95a45f-e8ec-402b-97eb-9089ed4c2bce-kube-api-access-rh7tn\") pod \"node-ca-w4lz2\" (UID: \"7c95a45f-e8ec-402b-97eb-9089ed4c2bce\") " pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.786483 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-etc-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786483 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-log-socket\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786483 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovnkube-config\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786520 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-log-socket\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovn-node-metrics-cert\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786535 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786499 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-etc-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4lgf\" (UniqueName: \"kubernetes.io/projected/21a14840-5dfc-4c8d-91b1-ef5db29807f6-kube-api-access-f4lgf\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786582 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786658 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-run-netns\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786709 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-env-overrides\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786730 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-socket-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.786752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-node-log\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-system-cni-dir\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786799 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.786815 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786824 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-sys\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786851 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-node-log\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786860 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw2st\" (UniqueName: \"kubernetes.io/projected/5af88a0e-a538-435c-a540-65e938e7ecf6-kube-api-access-gw2st\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786880 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-run-netns\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.786895 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:30.286874086 +0000 UTC m=+3.045945080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786909 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-system-cni-dir\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-dbus\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-device-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786970 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw7r6\" (UniqueName: \"kubernetes.io/projected/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-kube-api-access-bw7r6\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.787334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.786993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-systemd\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787003 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-socket-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-iptables-alerter-script\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787083 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysctl-d\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787116 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-slash\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-registration-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787161 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovnkube-script-lib\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787173 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovnkube-config\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787191 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787217 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787241 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-systemd\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787242 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-modprobe-d\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787346 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-env-overrides\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787384 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysctl-d\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.787972 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787418 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-sys\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787491 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-cni-bin\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787461 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-modprobe-d\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787520 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787546 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-device-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787587 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvsgl\" (UniqueName: \"kubernetes.io/projected/8e2f2115-998c-4843-bf33-1f7758ba633e-kube-api-access-zvsgl\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787599 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787612 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-systemd-units\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787637 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787655 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-dbus\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cnibin\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787679 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-iptables-alerter-script\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.788615 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787692 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-etc-selinux\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-slash\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-kubernetes\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787747 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-systemd-units\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787765 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-registration-dir\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787787 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-cni-bin\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787794 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-kubernetes\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787802 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-cnibin\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787852 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787867 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovnkube-script-lib\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787884 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-run\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-cni-netd\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787922 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787937 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787965 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-run\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.787947 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-cni-netd\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxr2n\" (UniqueName: \"kubernetes.io/projected/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-kube-api-access-zxr2n\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788038 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-lib-modules\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788072 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-tuned\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-kubelet-config\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2rnb\" (UniqueName: \"kubernetes.io/projected/372108ce-3670-4e52-9da3-6d8a0df670eb-kube-api-access-c2rnb\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788169 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-lib-modules\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-systemd\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788242 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-kubelet-config\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788247 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5af88a0e-a538-435c-a540-65e938e7ecf6-hosts-file\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788281 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5af88a0e-a538-435c-a540-65e938e7ecf6-hosts-file\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:29.789903 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.788360 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788405 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-systemd\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:29.788414 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:30.288399841 +0000 UTC m=+3.047470851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysconfig\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788456 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-os-release\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788485 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysctl-conf\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788511 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysconfig\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788519 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-sys-fs\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788561 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-kubelet\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788569 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/372108ce-3670-4e52-9da3-6d8a0df670eb-sys-fs\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788590 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-os-release\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788613 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-host-kubelet\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788624 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-ovn\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-run-ovn\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.790633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788664 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-host-slash\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788667 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-sysctl-conf\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788690 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-var-lib-kubelet\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-host-slash\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788721 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e2f2115-998c-4843-bf33-1f7758ba633e-tmp\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788747 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-var-lib-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788774 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkk5c\" (UniqueName: \"kubernetes.io/projected/1914f60a-ab07-4a57-8c3e-05cdcc59407c-kube-api-access-gkk5c\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788805 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-var-lib-kubelet\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788819 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1914f60a-ab07-4a57-8c3e-05cdcc59407c-var-lib-openvswitch\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af88a0e-a538-435c-a540-65e938e7ecf6-tmp-dir\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788879 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-host\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.788959 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e2f2115-998c-4843-bf33-1f7758ba633e-host\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.789138 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af88a0e-a538-435c-a540-65e938e7ecf6-tmp-dir\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.789651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1914f60a-ab07-4a57-8c3e-05cdcc59407c-ovn-node-metrics-cert\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.790133 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e2f2115-998c-4843-bf33-1f7758ba633e-etc-tuned\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.791149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.790921 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e2f2115-998c-4843-bf33-1f7758ba633e-tmp\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.796654 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.796633 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw2st\" (UniqueName: \"kubernetes.io/projected/5af88a0e-a538-435c-a540-65e938e7ecf6-kube-api-access-gw2st\") pod \"node-resolver-rqrtl\" (UID: \"5af88a0e-a538-435c-a540-65e938e7ecf6\") " pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.796896 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.796876 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4lgf\" (UniqueName: \"kubernetes.io/projected/21a14840-5dfc-4c8d-91b1-ef5db29807f6-kube-api-access-f4lgf\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:29.798846 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.798825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvsgl\" (UniqueName: \"kubernetes.io/projected/8e2f2115-998c-4843-bf33-1f7758ba633e-kube-api-access-zvsgl\") pod \"tuned-g6l4d\" (UID: \"8e2f2115-998c-4843-bf33-1f7758ba633e\") " pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.800577 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.800557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkk5c\" (UniqueName: \"kubernetes.io/projected/1914f60a-ab07-4a57-8c3e-05cdcc59407c-kube-api-access-gkk5c\") pod \"ovnkube-node-rlb2q\" (UID: \"1914f60a-ab07-4a57-8c3e-05cdcc59407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.800698 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.800659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2rnb\" (UniqueName: \"kubernetes.io/projected/372108ce-3670-4e52-9da3-6d8a0df670eb-kube-api-access-c2rnb\") pod \"aws-ebs-csi-driver-node-lgrbx\" (UID: \"372108ce-3670-4e52-9da3-6d8a0df670eb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.801066 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.801040 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxr2n\" (UniqueName: \"kubernetes.io/projected/4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1-kube-api-access-zxr2n\") pod \"multus-additional-cni-plugins-rlmqv\" (UID: \"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1\") " pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.801953 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.801936 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw7r6\" (UniqueName: \"kubernetes.io/projected/b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab-kube-api-access-bw7r6\") pod \"iptables-alerter-f94zf\" (UID: \"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab\") " pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.874883 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.874845 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:29.880361 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.880252 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w4lz2" Jul 23 21:51:29.887624 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.887451 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ntkwx" Jul 23 21:51:29.892788 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.892572 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f94zf" Jul 23 21:51:29.892909 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.892885 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c95a45f_e8ec_402b_97eb_9089ed4c2bce.slice/crio-126b79ee4113bbc24e69f844b3f1b888705953a2dcfbd387e2690313ca0369d1 WatchSource:0}: Error finding container 126b79ee4113bbc24e69f844b3f1b888705953a2dcfbd387e2690313ca0369d1: Status 404 returned error can't find the container with id 126b79ee4113bbc24e69f844b3f1b888705953a2dcfbd387e2690313ca0369d1 Jul 23 21:51:29.899027 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.898995 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11144f63_f9a8_4fcc_a309_56aee07f6423.slice/crio-43b348dc79c8426cfa4213989a672d936e463e2516655c840d703887ffc0f38c WatchSource:0}: Error finding container 43b348dc79c8426cfa4213989a672d936e463e2516655c840d703887ffc0f38c: Status 404 returned error can't find the container with id 43b348dc79c8426cfa4213989a672d936e463e2516655c840d703887ffc0f38c Jul 23 21:51:29.899240 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.899089 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rqrtl" Jul 23 21:51:29.905072 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.904756 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" Jul 23 21:51:29.905147 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.905092 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d3adac_57d9_4ab7_9d3b_ebf4ee786eab.slice/crio-0af71bf9921dc5f9a0e583e49637eae7204d8dcf4627e17a80de8a24e02671fe WatchSource:0}: Error finding container 0af71bf9921dc5f9a0e583e49637eae7204d8dcf4627e17a80de8a24e02671fe: Status 404 returned error can't find the container with id 0af71bf9921dc5f9a0e583e49637eae7204d8dcf4627e17a80de8a24e02671fe Jul 23 21:51:29.910543 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.910518 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" Jul 23 21:51:29.915269 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.915199 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:29.919634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:29.919609 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" Jul 23 21:51:29.919805 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.919776 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372108ce_3670_4e52_9da3_6d8a0df670eb.slice/crio-2aa3d20169ee55b710fc41950cd30e7e2f65914bf0558d2d74c58c5300e742cb WatchSource:0}: Error finding container 2aa3d20169ee55b710fc41950cd30e7e2f65914bf0558d2d74c58c5300e742cb: Status 404 returned error can't find the container with id 2aa3d20169ee55b710fc41950cd30e7e2f65914bf0558d2d74c58c5300e742cb Jul 23 21:51:29.930742 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.930711 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2f2115_998c_4843_bf33_1f7758ba633e.slice/crio-9ad1234c3ecc3db48646cf82ea1d0f16ce0625d94e6302dc02e56d7a5b7382c2 WatchSource:0}: Error finding container 9ad1234c3ecc3db48646cf82ea1d0f16ce0625d94e6302dc02e56d7a5b7382c2: Status 404 returned error can't find the container with id 9ad1234c3ecc3db48646cf82ea1d0f16ce0625d94e6302dc02e56d7a5b7382c2 Jul 23 21:51:29.934957 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.934928 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1914f60a_ab07_4a57_8c3e_05cdcc59407c.slice/crio-db6f41a8117c790fc4d7591046f5b9e8c492a06bc5f98384e4c77f7ab14fa226 WatchSource:0}: Error finding container db6f41a8117c790fc4d7591046f5b9e8c492a06bc5f98384e4c77f7ab14fa226: Status 404 returned error can't find the container with id db6f41a8117c790fc4d7591046f5b9e8c492a06bc5f98384e4c77f7ab14fa226 Jul 23 21:51:29.936722 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:51:29.936690 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4f2c9d_6f0f_43a2_8ad9_35dfd93493b1.slice/crio-7f1ca85370a62365da745bd78b5b422bb7f9c7c8ac5b867d8a999be8128039a9 WatchSource:0}: Error finding container 7f1ca85370a62365da745bd78b5b422bb7f9c7c8ac5b867d8a999be8128039a9: Status 404 returned error can't find the container with id 7f1ca85370a62365da745bd78b5b422bb7f9c7c8ac5b867d8a999be8128039a9 Jul 23 21:51:30.293216 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.293174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:30.293428 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.293239 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:30.293428 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.293284 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:30.293428 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.293404 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:30.293648 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.293467 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:31.293449168 +0000 UTC m=+4.052520177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:30.293940 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.293919 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:30.294017 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.293948 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:30.294017 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.293962 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:30.294017 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.294011 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:31.293994991 +0000 UTC m=+4.053065985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:30.294254 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.294093 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:30.294254 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:30.294129 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:31.294116527 +0000 UTC m=+4.053187519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:30.640218 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.640084 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 21:46:29 +0000 UTC" deadline="2028-03-28 20:03:10.881085276 +0000 UTC" Jul 23 21:51:30.640218 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.640144 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14734h11m40.240945406s" Jul 23 21:51:30.701553 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.701380 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"db6f41a8117c790fc4d7591046f5b9e8c492a06bc5f98384e4c77f7ab14fa226"} Jul 23 21:51:30.708401 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.708340 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" event={"ID":"8e2f2115-998c-4843-bf33-1f7758ba633e","Type":"ContainerStarted","Data":"9ad1234c3ecc3db48646cf82ea1d0f16ce0625d94e6302dc02e56d7a5b7382c2"} Jul 23 21:51:30.718212 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.718135 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rqrtl" event={"ID":"5af88a0e-a538-435c-a540-65e938e7ecf6","Type":"ContainerStarted","Data":"3f0537cf20823dadf39de1d909bbced91a2fa988f277a7e202bf8fca970077d1"} Jul 23 21:51:30.722368 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.722269 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f94zf" event={"ID":"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab","Type":"ContainerStarted","Data":"0af71bf9921dc5f9a0e583e49637eae7204d8dcf4627e17a80de8a24e02671fe"} Jul 23 21:51:30.731438 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.729988 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerStarted","Data":"7f1ca85370a62365da745bd78b5b422bb7f9c7c8ac5b867d8a999be8128039a9"} Jul 23 21:51:30.733961 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.733881 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" event={"ID":"372108ce-3670-4e52-9da3-6d8a0df670eb","Type":"ContainerStarted","Data":"2aa3d20169ee55b710fc41950cd30e7e2f65914bf0558d2d74c58c5300e742cb"} Jul 23 21:51:30.739364 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.739292 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ntkwx" event={"ID":"11144f63-f9a8-4fcc-a309-56aee07f6423","Type":"ContainerStarted","Data":"43b348dc79c8426cfa4213989a672d936e463e2516655c840d703887ffc0f38c"} Jul 23 21:51:30.744236 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.744204 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w4lz2" event={"ID":"7c95a45f-e8ec-402b-97eb-9089ed4c2bce","Type":"ContainerStarted","Data":"126b79ee4113bbc24e69f844b3f1b888705953a2dcfbd387e2690313ca0369d1"} Jul 23 21:51:30.747696 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:30.747663 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mrck5" event={"ID":"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09","Type":"ContainerStarted","Data":"f38b056719b4287937ddce7133d4dbba9816b9b3e52d6cd2d605b0ab2cf8fc91"} Jul 23 21:51:31.303205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.303166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:31.303399 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.303224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:31.303399 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.303270 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:31.303507 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.303405 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:31.303507 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.303468 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:33.30344987 +0000 UTC m=+6.062520878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:31.303900 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.303877 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:31.303900 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.303903 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:31.304080 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.303915 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:31.304080 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.303961 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:33.303946423 +0000 UTC m=+6.063017425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:31.304080 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.304021 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:31.304223 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.304070 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:33.304044208 +0000 UTC m=+6.063115201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:31.641217 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.641171 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 21:46:29 +0000 UTC" deadline="2028-04-16 21:04:12.130428486 +0000 UTC" Jul 23 21:51:31.641217 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.641215 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15191h12m40.489216992s" Jul 23 21:51:31.674905 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.674872 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:31.675087 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.674999 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:31.675429 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.675283 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:31.675429 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:31.675309 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:31.675429 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.675382 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:31.675645 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:31.675482 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:33.326412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:33.326465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:33.326507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326595 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326627 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326607 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326669 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:37.326650638 +0000 UTC m=+10.085721630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326686 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:33.326685 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326702 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:33.327579 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326689 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:37.326679324 +0000 UTC m=+10.085750317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:33.327579 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.326750 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:37.326738251 +0000 UTC m=+10.085809239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:33.672295 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:33.672262 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:33.672295 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:33.672286 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:33.672552 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:33.672262 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:33.672552 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.672412 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:33.672552 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.672536 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:33.672708 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:33.672623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:35.673383 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:35.672620 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:35.673383 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:35.672766 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:35.673383 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:35.673227 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:35.673383 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:35.673332 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:35.674230 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:35.674100 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:35.674230 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:35.674194 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:37.360674 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:37.360633 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:37.361171 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:37.360727 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:37.361171 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:37.360762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:37.361171 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.360888 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:37.361171 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.360954 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:45.360933585 +0000 UTC m=+18.120004577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:37.361408 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.361372 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:37.361456 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.361429 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:45.361413674 +0000 UTC m=+18.120484667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:37.361543 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.361514 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:37.361543 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.361534 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:37.361640 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.361555 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:37.361640 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.361598 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:51:45.361586033 +0000 UTC m=+18.120657027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:37.673739 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:37.673662 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:37.673897 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.673767 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:37.673897 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:37.673822 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:37.674007 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.673922 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:37.674007 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:37.673937 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:37.674123 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:37.674095 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:39.672877 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:39.672369 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:39.672877 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:39.672367 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:39.672877 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:39.672514 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:39.672877 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:39.672627 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:39.672877 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:39.672677 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:39.672877 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:39.672750 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:41.673208 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:41.673169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:41.673671 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:41.673169 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:41.673671 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:41.673318 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:41.673671 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:41.673368 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:41.673671 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:41.673182 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:41.673671 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:41.673452 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:43.672888 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:43.672853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:43.673353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:43.672860 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:43.673353 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:43.672985 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:43.673353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:43.672861 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:43.673353 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:43.673086 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:43.673353 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:43.673214 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:45.425636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:45.425595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:45.425636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:45.425638 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:45.425669 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425775 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425804 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425826 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425832 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:52:01.425817842 +0000 UTC m=+34.184888831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425835 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425773 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425885 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:52:01.425869219 +0000 UTC m=+34.184940208 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:45.426228 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.425906 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:01.425895955 +0000 UTC m=+34.184966978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:45.672699 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:45.672659 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:45.672850 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:45.672665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:45.672850 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.672797 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:45.672935 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.672878 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:45.672935 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:45.672665 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:45.672998 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:45.672950 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:47.673539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:47.673501 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:47.673539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:47.673540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:47.674088 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:47.673623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:47.674088 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:47.673710 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:47.674088 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:47.673762 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:47.674088 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:47.673835 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:49.673148 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.672965 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:49.673539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.672962 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:49.673539 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:49.673258 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:49.673539 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:49.673344 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:49.673539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.673019 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:49.673539 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:49.673432 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:49.796346 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.796135 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" event={"ID":"634343fa5cba8d6321e15d7ff451094e","Type":"ContainerStarted","Data":"5d61400a21fd6696f0bc2ae19f12b442321a29f6add50f051fd415bcdb0c01c1"} Jul 23 21:51:49.798770 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.798746 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ntkwx" event={"ID":"11144f63-f9a8-4fcc-a309-56aee07f6423","Type":"ContainerStarted","Data":"e285768b51bc4cd9945be9c9ac39bbc2eaeb99aeaf697331f6dc22db0d1a0de8"} Jul 23 21:51:49.801126 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.801107 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:51:49.801506 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.801486 2579 generic.go:356] "Generic (PLEG): container finished" podID="1914f60a-ab07-4a57-8c3e-05cdcc59407c" containerID="09d62166fe7da0a2abc597fad436ce2d319017ac0c26d7c8c9ded27b0a867615" exitCode=1 Jul 23 21:51:49.801592 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.801554 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"ebe1386164710c869bd27301b1cba9ae8b2d16b9a509a4a3aadf1807bf46cf9c"} Jul 23 21:51:49.801659 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.801615 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerDied","Data":"09d62166fe7da0a2abc597fad436ce2d319017ac0c26d7c8c9ded27b0a867615"} Jul 23 21:51:49.801713 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.801659 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"a33d46d034c473672331991bc9dff1cf8fe5301a6b2f672c697d45b613789599"} Jul 23 21:51:49.803143 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.803118 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" event={"ID":"8e2f2115-998c-4843-bf33-1f7758ba633e","Type":"ContainerStarted","Data":"9525dd61b92409d9be547bf8775b17babd9a53c135fd1b2767e7dfe58a66596b"} Jul 23 21:51:49.812445 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.812406 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-32.ec2.internal" podStartSLOduration=20.812392684 podStartE2EDuration="20.812392684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:49.811848721 +0000 UTC m=+22.570919731" watchObservedRunningTime="2026-07-23 21:51:49.812392684 +0000 UTC m=+22.571463694" Jul 23 21:51:49.833086 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.832003 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ntkwx" podStartSLOduration=3.229992631 podStartE2EDuration="22.831983923s" totalImagesPullingTime="19.601991292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:49.831102885 +0000 UTC m=+22.590173921" watchObservedRunningTime="2026-07-23 21:51:49.831983923 +0000 UTC m=+22.591054938" Jul 23 21:51:49.852069 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:49.852017 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g6l4d" podStartSLOduration=2.387000541 podStartE2EDuration="21.851999637s" totalImagesPullingTime="19.464999096s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:49.851527919 +0000 UTC m=+22.610598927" watchObservedRunningTime="2026-07-23 21:51:49.851999637 +0000 UTC m=+22.611070647" Jul 23 21:51:50.806768 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.806738 2579 generic.go:356] "Generic (PLEG): container finished" podID="78661e095b3e65c966e2cbaf1488640c" containerID="e47a5483ea3f20a44ef0e2046220b6e66a1fb7c96b8cbdef9c435f60471f433e" exitCode=0 Jul 23 21:51:50.807629 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.806833 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" event={"ID":"78661e095b3e65c966e2cbaf1488640c","Type":"ContainerDied","Data":"e47a5483ea3f20a44ef0e2046220b6e66a1fb7c96b8cbdef9c435f60471f433e"} Jul 23 21:51:50.810111 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.810085 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:51:50.810584 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.810562 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"bb3c149ad9461b67bd452838e14f05f9f6cd7276f5b2e597de2bd6608e95573d"} Jul 23 21:51:50.810667 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.810628 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"c4ab31f2f8ceab1c86a1366a4214521c45ad01bde7b7676f1e3123531c8f71de"} Jul 23 21:51:50.810711 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.810665 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"bd44862253a09b896484c3a0f821c17d7d402f82eb13a93625dd7d7cbbd18f09"} Jul 23 21:51:50.812383 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.812356 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rqrtl" event={"ID":"5af88a0e-a538-435c-a540-65e938e7ecf6","Type":"ContainerStarted","Data":"2e308e4fabfd958ef807b1688a0082af852dd65db64df5f5ce795aed6b45df86"} Jul 23 21:51:50.813968 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.813931 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f94zf" event={"ID":"b5d3adac-57d9-4ab7-9d3b-ebf4ee786eab","Type":"ContainerStarted","Data":"abea9c40f976d66926a43bd8d671bb4661ed9cce35372e0e8aaea225022827aa"} Jul 23 21:51:50.815636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.815613 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1" containerID="70d0efe5e8dcc4cf89a6fcd0fa0934d1d63670637ef5edda62b3852b26ec48ac" exitCode=0 Jul 23 21:51:50.815752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.815686 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerDied","Data":"70d0efe5e8dcc4cf89a6fcd0fa0934d1d63670637ef5edda62b3852b26ec48ac"} Jul 23 21:51:50.817368 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.817338 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" event={"ID":"372108ce-3670-4e52-9da3-6d8a0df670eb","Type":"ContainerStarted","Data":"abd74e24eaeb0c9c1fef1b3648d5e36716d726f0c3cec626492babc4e6b51006"} Jul 23 21:51:50.819099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.819038 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w4lz2" event={"ID":"7c95a45f-e8ec-402b-97eb-9089ed4c2bce","Type":"ContainerStarted","Data":"6be8bd44fc45eae18729bdc9745d6f6c6282dbb37f1259afc5ffa8f94615c172"} Jul 23 21:51:50.820503 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.820481 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mrck5" event={"ID":"f35f40c2-adf4-4b5c-831f-c9b9f0c40b09","Type":"ContainerStarted","Data":"d45e2da594b6ec846f6c36a284d0fc37b7b139be8b6869578f02c9aefe92ce09"} Jul 23 21:51:50.828739 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.828693 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rqrtl" podStartSLOduration=4.346745577 podStartE2EDuration="23.828669605s" totalImagesPullingTime="19.481924028s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:50.828153615 +0000 UTC m=+23.587224628" watchObservedRunningTime="2026-07-23 21:51:50.828669605 +0000 UTC m=+23.587740618" Jul 23 21:51:50.872287 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.872248 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f94zf" podStartSLOduration=4.41114448 podStartE2EDuration="23.872232917s" totalImagesPullingTime="19.461088437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:50.847117738 +0000 UTC m=+23.606188778" watchObservedRunningTime="2026-07-23 21:51:50.872232917 +0000 UTC m=+23.631303928" Jul 23 21:51:50.888414 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:50.888362 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mrck5" podStartSLOduration=4.382110627 podStartE2EDuration="23.888343551s" totalImagesPullingTime="19.506232924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:50.887982929 +0000 UTC m=+23.647053940" watchObservedRunningTime="2026-07-23 21:51:50.888343551 +0000 UTC m=+23.647414564" Jul 23 21:51:51.641008 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.640794 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 21:51:51.653639 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.653559 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T21:51:51.640998313Z","UUID":"85dcb71e-40e1-4aaf-a7b0-38e7215aa4f1","Handler":null,"Name":"","Endpoint":""} Jul 23 21:51:51.655129 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.655108 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 21:51:51.655220 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.655153 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 21:51:51.673273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.673228 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:51.673381 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.673235 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:51.673381 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:51.673346 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:51.673495 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:51.673447 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:51.673548 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.673498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:51.673599 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:51.673563 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:51.825109 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.824955 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" event={"ID":"372108ce-3670-4e52-9da3-6d8a0df670eb","Type":"ContainerStarted","Data":"a5a9df6cf2de2eb7a0dffc08eed7734fc4642a2213e8c665f5fe42331107cfff"} Jul 23 21:51:51.841738 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:51.841698 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w4lz2" podStartSLOduration=5.340675312 podStartE2EDuration="24.84168199s" totalImagesPullingTime="19.501006678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:50.903686899 +0000 UTC m=+23.662757913" watchObservedRunningTime="2026-07-23 21:51:51.84168199 +0000 UTC m=+24.600753004" Jul 23 21:51:52.828845 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:52.828807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" event={"ID":"78661e095b3e65c966e2cbaf1488640c","Type":"ContainerStarted","Data":"3624634ae56ce39150ec32d12c29de8c82782304ed049d184d510b449e25f1cd"} Jul 23 21:51:52.831609 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:52.831588 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:51:52.831945 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:52.831926 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"0880b2960adaa8cf8d4e5f9fdb0d3caf5f43536b490970b88fc6c657a2e330de"} Jul 23 21:51:52.845288 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:52.845235 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-32.ec2.internal" podStartSLOduration=23.845221402 podStartE2EDuration="23.845221402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:52.844797617 +0000 UTC m=+25.603868629" watchObservedRunningTime="2026-07-23 21:51:52.845221402 +0000 UTC m=+25.604292412" Jul 23 21:51:53.090027 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.089944 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:53.090700 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.090670 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:53.673199 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.673160 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:53.673407 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.673217 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:53.673407 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.673223 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:53.673407 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:53.673310 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:53.673566 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:53.673451 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:53.673566 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:53.673536 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:53.836836 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.836792 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" event={"ID":"372108ce-3670-4e52-9da3-6d8a0df670eb","Type":"ContainerStarted","Data":"819c0a9a95275ed442f6c3ac1a4ce93e7c785a6d328f7541dc08b982c2782011"} Jul 23 21:51:53.837388 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.837087 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:53.837682 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.837659 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mrck5" Jul 23 21:51:53.873810 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:53.873765 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lgrbx" podStartSLOduration=3.947828684 podStartE2EDuration="25.873749689s" totalImagesPullingTime="21.925921005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:53.858776933 +0000 UTC m=+26.617847958" watchObservedRunningTime="2026-07-23 21:51:53.873749689 +0000 UTC m=+26.632820700" Jul 23 21:51:55.672347 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:55.672311 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:55.672695 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:55.672420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:55.672695 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:55.672430 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:55.672695 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:55.672523 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:55.672695 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:55.672572 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:55.672695 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:55.672623 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:55.844413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:55.844392 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:51:55.844822 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:55.844794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"c1b161b0c87d98a879877fc4559e08711364358261aa1908c0cc974c50c18974"} Jul 23 21:51:55.845911 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:55.845884 2579 scope.go:117] "RemoveContainer" containerID="09d62166fe7da0a2abc597fad436ce2d319017ac0c26d7c8c9ded27b0a867615" Jul 23 21:51:56.851315 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.850280 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1" containerID="eb74dda01d3cc294be55ffb4fc4773017f7a2e404f7de322503beddc2755eeef" exitCode=0 Jul 23 21:51:56.851315 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.850378 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerDied","Data":"eb74dda01d3cc294be55ffb4fc4773017f7a2e404f7de322503beddc2755eeef"} Jul 23 21:51:56.856233 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.855813 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:51:56.856308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.856253 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" event={"ID":"1914f60a-ab07-4a57-8c3e-05cdcc59407c","Type":"ContainerStarted","Data":"bc44f5c53277d6a8f46b03a14b380f0b16253d08785962b70e4daaf75f48d9c9"} Jul 23 21:51:56.857367 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.857341 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:56.857489 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.857475 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:56.857547 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.857537 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:56.881453 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.881426 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:56.884536 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.882197 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:51:56.910633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:56.910592 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" podStartSLOduration=9.432266936 podStartE2EDuration="28.91057812s" totalImagesPullingTime="19.478311184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:56.908646888 +0000 UTC m=+29.667717936" watchObservedRunningTime="2026-07-23 21:51:56.91057812 +0000 UTC m=+29.669649131" Jul 23 21:51:57.672976 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.672948 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:57.673140 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.673064 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:57.673140 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:57.673077 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:57.673215 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:57.673151 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:57.673215 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.673175 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:57.673276 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:57.673246 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:57.806753 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.806720 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mgtn9"] Jul 23 21:51:57.810428 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.810400 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gp9l9"] Jul 23 21:51:57.823632 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.823601 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kncqt"] Jul 23 21:51:57.858734 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.858713 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:57.858734 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.858723 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:57.859326 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:57.858720 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:57.859326 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:57.858825 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:51:57.859326 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:57.858891 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:57.859326 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:57.858958 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:58.864290 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:58.864072 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1" containerID="f765e493ca57b08512814a9eaeb95a8d68e231771662ee41bb1cacf78acc36a0" exitCode=0 Jul 23 21:51:58.864647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:58.864147 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerDied","Data":"f765e493ca57b08512814a9eaeb95a8d68e231771662ee41bb1cacf78acc36a0"} Jul 23 21:51:59.672564 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:59.672531 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:51:59.672714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:59.672540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:51:59.672714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:51:59.672661 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:51:59.672714 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:59.672659 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:51:59.672824 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:59.672773 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:51:59.672869 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:51:59.672843 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:52:00.871205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:00.871175 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1" containerID="0d479f6c29f10aa4a4fc6f6c01c6fcc9a4585df1f1da999b33a626d8c3573874" exitCode=0 Jul 23 21:52:00.871699 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:00.871219 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerDied","Data":"0d479f6c29f10aa4a4fc6f6c01c6fcc9a4585df1f1da999b33a626d8c3573874"} Jul 23 21:52:01.448976 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:01.448945 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:52:01.449173 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:01.448998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:01.449173 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:01.449018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:52:01.449173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449096 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:52:01.449173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449121 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:52:01.449173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449155 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret podName:ac4ced1f-d98f-4f63-94bd-8dd118c70c3e nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.449140263 +0000 UTC m=+66.208211252 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret") pod "global-pull-secret-syncer-mgtn9" (UID: "ac4ced1f-d98f-4f63-94bd-8dd118c70c3e") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:52:01.449173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449171 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs podName:21a14840-5dfc-4c8d-91b1-ef5db29807f6 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.449163165 +0000 UTC m=+66.208234153 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs") pod "network-metrics-daemon-gp9l9" (UID: "21a14840-5dfc-4c8d-91b1-ef5db29807f6") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:52:01.449448 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449154 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:52:01.449448 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449254 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:52:01.449448 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449272 2579 projected.go:196] Error preparing data for projected volume kube-api-access-6wd2m for pod openshift-network-diagnostics/network-check-target-kncqt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:52:01.449448 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.449325 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m podName:c25b748b-2ddd-40fc-9127-4c8cdd8ac42e nodeName:}" failed. No retries permitted until 2026-07-23 21:52:33.449308523 +0000 UTC m=+66.208379523 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6wd2m" (UniqueName: "kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m") pod "network-check-target-kncqt" (UID: "c25b748b-2ddd-40fc-9127-4c8cdd8ac42e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:52:01.672696 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:01.672656 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:52:01.672696 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:01.672695 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:01.672995 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:01.672656 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:52:01.672995 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.672788 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-gp9l9" podUID="21a14840-5dfc-4c8d-91b1-ef5db29807f6" Jul 23 21:52:01.672995 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.672850 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mgtn9" podUID="ac4ced1f-d98f-4f63-94bd-8dd118c70c3e" Jul 23 21:52:01.672995 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:01.672919 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kncqt" podUID="c25b748b-2ddd-40fc-9127-4c8cdd8ac42e" Jul 23 21:52:02.548754 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.548722 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-32.ec2.internal" event="NodeReady" Jul 23 21:52:02.549174 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.548870 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 21:52:02.585049 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.585018 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-765678454c-xhg9v"] Jul 23 21:52:02.589393 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.589365 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.592163 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.592139 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 21:52:02.592291 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.592176 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 21:52:02.592360 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.592290 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hkh5w\"" Jul 23 21:52:02.592360 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.592299 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 21:52:02.598071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.598036 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 21:52:02.602620 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.602595 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-765678454c-xhg9v"] Jul 23 21:52:02.604919 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.604898 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l4gl4"] Jul 23 21:52:02.608757 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.608736 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-966bw"] Jul 23 21:52:02.608930 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.608858 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.611548 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.611530 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 21:52:02.611643 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.611564 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 21:52:02.611847 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.611832 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq952\"" Jul 23 21:52:02.612291 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.612276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.616466 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.616023 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 21:52:02.616466 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.616459 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swdkn\"" Jul 23 21:52:02.616723 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.616704 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 21:52:02.616930 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.616889 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 21:52:02.620564 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.620543 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l4gl4"] Jul 23 21:52:02.622791 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.622772 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-966bw"] Jul 23 21:52:02.749356 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.749323 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrck5_f35f40c2-adf4-4b5c-831f-c9b9f0c40b09/konnectivity-agent/0.log" limit=1024 Jul 23 21:52:02.760334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760293 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gp7x\" (UniqueName: \"kubernetes.io/projected/baf45753-a888-4811-b238-767035037d2e-kube-api-access-2gp7x\") pod \"ingress-canary-966bw\" (UID: \"baf45753-a888-4811-b238-767035037d2e\") " pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.760469 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760469 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-tmp-dir\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.760469 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760447 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-image-registry-private-configuration\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-trusted-ca\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760583 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-certificates\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760733 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760712 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-installation-pull-secrets\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760766 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760744 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtcbx\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-kube-api-access-gtcbx\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760810 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760781 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-config-volume\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.760845 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84b492fd-77e2-4ce2-9a18-5391c94b98d5-ca-trust-extracted\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760884 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf45753-a888-4811-b238-767035037d2e-cert\") pod \"ingress-canary-966bw\" (UID: \"baf45753-a888-4811-b238-767035037d2e\") " pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.760884 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760875 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-bound-sa-token\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.760953 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.760953 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.760931 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24lps\" (UniqueName: \"kubernetes.io/projected/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-kube-api-access-24lps\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.847034 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.846953 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrck5_f35f40c2-adf4-4b5c-831f-c9b9f0c40b09/konnectivity-agent/0.log" limit=1024 Jul 23 21:52:02.862101 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-certificates\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862263 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862108 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-installation-pull-secrets\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862263 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862125 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcbx\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-kube-api-access-gtcbx\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862263 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862141 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-config-volume\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.862263 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84b492fd-77e2-4ce2-9a18-5391c94b98d5-ca-trust-extracted\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862511 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862488 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf45753-a888-4811-b238-767035037d2e-cert\") pod \"ingress-canary-966bw\" (UID: \"baf45753-a888-4811-b238-767035037d2e\") " pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.862577 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-bound-sa-token\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862577 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.862668 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862584 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84b492fd-77e2-4ce2-9a18-5391c94b98d5-ca-trust-extracted\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862668 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24lps\" (UniqueName: \"kubernetes.io/projected/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-kube-api-access-24lps\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.862746 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gp7x\" (UniqueName: \"kubernetes.io/projected/baf45753-a888-4811-b238-767035037d2e-kube-api-access-2gp7x\") pod \"ingress-canary-966bw\" (UID: \"baf45753-a888-4811-b238-767035037d2e\") " pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.862746 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:02.862676 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:02.862746 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862746 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:02.862727 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls podName:4de109c8-8b50-46f2-b7ae-e4340a11cdcf nodeName:}" failed. No retries permitted until 2026-07-23 21:52:03.362709266 +0000 UTC m=+36.121780266 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls") pod "dns-default-l4gl4" (UID: "4de109c8-8b50-46f2-b7ae-e4340a11cdcf") : secret "dns-default-metrics-tls" not found Jul 23 21:52:02.862746 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-config-volume\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862755 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-tmp-dir\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:02.862769 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:02.862780 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765678454c-xhg9v: secret "image-registry-tls" not found Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862800 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-image-registry-private-configuration\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:02.862814 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls podName:84b492fd-77e2-4ce2-9a18-5391c94b98d5 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:03.362802364 +0000 UTC m=+36.121873356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls") pod "image-registry-765678454c-xhg9v" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5") : secret "image-registry-tls" not found Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-trusted-ca\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.862965 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.862765 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-certificates\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.863291 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.863138 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-tmp-dir\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.863653 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.863632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-trusted-ca\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.867019 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.866996 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-image-registry-private-configuration\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.867127 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.867002 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-installation-pull-secrets\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.867127 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.867069 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/baf45753-a888-4811-b238-767035037d2e-cert\") pod \"ingress-canary-966bw\" (UID: \"baf45753-a888-4811-b238-767035037d2e\") " pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.874650 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.874396 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gp7x\" (UniqueName: \"kubernetes.io/projected/baf45753-a888-4811-b238-767035037d2e-kube-api-access-2gp7x\") pod \"ingress-canary-966bw\" (UID: \"baf45753-a888-4811-b238-767035037d2e\") " pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:02.874798 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.874776 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtcbx\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-kube-api-access-gtcbx\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.875546 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.875524 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-bound-sa-token\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:02.875756 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.875739 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24lps\" (UniqueName: \"kubernetes.io/projected/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-kube-api-access-24lps\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:02.926565 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:02.926532 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-966bw" Jul 23 21:52:03.080620 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.080394 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-966bw"] Jul 23 21:52:03.083963 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:03.083936 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf45753_a888_4811_b238_767035037d2e.slice/crio-a5ffe3055ef1813cbb9a83874455175ac979876adceac1b229fa7312907a58d3 WatchSource:0}: Error finding container a5ffe3055ef1813cbb9a83874455175ac979876adceac1b229fa7312907a58d3: Status 404 returned error can't find the container with id a5ffe3055ef1813cbb9a83874455175ac979876adceac1b229fa7312907a58d3 Jul 23 21:52:03.366494 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.366462 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:03.366672 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.366540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:03.366672 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:03.366629 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:03.366772 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:03.366701 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls podName:4de109c8-8b50-46f2-b7ae-e4340a11cdcf nodeName:}" failed. No retries permitted until 2026-07-23 21:52:04.366681559 +0000 UTC m=+37.125752550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls") pod "dns-default-l4gl4" (UID: "4de109c8-8b50-46f2-b7ae-e4340a11cdcf") : secret "dns-default-metrics-tls" not found Jul 23 21:52:03.366772 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:03.366701 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:03.366772 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:03.366722 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765678454c-xhg9v: secret "image-registry-tls" not found Jul 23 21:52:03.366772 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:03.366747 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls podName:84b492fd-77e2-4ce2-9a18-5391c94b98d5 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:04.366740529 +0000 UTC m=+37.125811550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls") pod "image-registry-765678454c-xhg9v" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5") : secret "image-registry-tls" not found Jul 23 21:52:03.672485 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.672410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:52:03.672485 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.672450 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:03.673250 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.672410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:52:03.678384 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.678364 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 21:52:03.679848 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.679822 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxtft\"" Jul 23 21:52:03.679945 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.679883 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 21:52:03.680067 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.680036 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9zwh\"" Jul 23 21:52:03.680146 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.680040 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 21:52:03.680199 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.680156 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 21:52:03.848347 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.848308 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrck5_f35f40c2-adf4-4b5c-831f-c9b9f0c40b09/konnectivity-agent/0.log" limit=1024 Jul 23 21:52:03.880334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:03.880302 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-966bw" event={"ID":"baf45753-a888-4811-b238-767035037d2e","Type":"ContainerStarted","Data":"a5ffe3055ef1813cbb9a83874455175ac979876adceac1b229fa7312907a58d3"} Jul 23 21:52:04.375992 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:04.375960 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:04.376177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:04.376033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:04.376177 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:04.376147 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:04.376304 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:04.376205 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:04.376304 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:04.376221 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls podName:4de109c8-8b50-46f2-b7ae-e4340a11cdcf nodeName:}" failed. No retries permitted until 2026-07-23 21:52:06.376204257 +0000 UTC m=+39.135275249 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls") pod "dns-default-l4gl4" (UID: "4de109c8-8b50-46f2-b7ae-e4340a11cdcf") : secret "dns-default-metrics-tls" not found Jul 23 21:52:04.376304 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:04.376223 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765678454c-xhg9v: secret "image-registry-tls" not found Jul 23 21:52:04.376304 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:04.376266 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls podName:84b492fd-77e2-4ce2-9a18-5391c94b98d5 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:06.376254383 +0000 UTC m=+39.135325371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls") pod "image-registry-765678454c-xhg9v" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5") : secret "image-registry-tls" not found Jul 23 21:52:06.390580 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:06.390407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:06.391090 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:06.390629 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:06.391090 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:06.390557 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:06.391090 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:06.390667 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765678454c-xhg9v: secret "image-registry-tls" not found Jul 23 21:52:06.391090 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:06.390719 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls podName:84b492fd-77e2-4ce2-9a18-5391c94b98d5 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:10.390701991 +0000 UTC m=+43.149772980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls") pod "image-registry-765678454c-xhg9v" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5") : secret "image-registry-tls" not found Jul 23 21:52:06.391090 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:06.390731 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:06.391090 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:06.390784 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls podName:4de109c8-8b50-46f2-b7ae-e4340a11cdcf nodeName:}" failed. No retries permitted until 2026-07-23 21:52:10.390759685 +0000 UTC m=+43.149830674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls") pod "dns-default-l4gl4" (UID: "4de109c8-8b50-46f2-b7ae-e4340a11cdcf") : secret "dns-default-metrics-tls" not found Jul 23 21:52:06.889129 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:06.889090 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-966bw" event={"ID":"baf45753-a888-4811-b238-767035037d2e","Type":"ContainerStarted","Data":"64cba27bce219625d2b36ca8e720cdea30b1084cd57cc9ca20366a4f182e8c81"} Jul 23 21:52:06.891684 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:06.891661 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1" containerID="498cc8f9d51fb3081c374af128d5a5cea00e1ec45904699383df6903c7328951" exitCode=0 Jul 23 21:52:06.891794 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:06.891712 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerDied","Data":"498cc8f9d51fb3081c374af128d5a5cea00e1ec45904699383df6903c7328951"} Jul 23 21:52:06.906845 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:06.906799 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-966bw" podStartSLOduration=1.902620603 podStartE2EDuration="4.90678416s" totalImagesPullingTime="3.004163557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:06.906101894 +0000 UTC m=+39.665172902" watchObservedRunningTime="2026-07-23 21:52:06.90678416 +0000 UTC m=+39.665855151" Jul 23 21:52:07.898582 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:07.898546 2579 generic.go:356] "Generic (PLEG): container finished" podID="4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1" containerID="d4dd9428d5ce81e7cd8e2fe09e5531267a11c7c31b7667c8935cc3fc23d7d17b" exitCode=0 Jul 23 21:52:07.899008 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:07.898642 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerDied","Data":"d4dd9428d5ce81e7cd8e2fe09e5531267a11c7c31b7667c8935cc3fc23d7d17b"} Jul 23 21:52:07.919172 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:07.919140 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrck5_f35f40c2-adf4-4b5c-831f-c9b9f0c40b09/konnectivity-agent/0.log" limit=1024 Jul 23 21:52:08.020212 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:08.020178 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mrck5_f35f40c2-adf4-4b5c-831f-c9b9f0c40b09/konnectivity-agent/0.log" limit=1024 Jul 23 21:52:08.904103 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:08.904067 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" event={"ID":"4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1","Type":"ContainerStarted","Data":"0f9d580b045193b7772de5f372d34c15e52be56c9b3e4e8ce0d2e36d6dba96b8"} Jul 23 21:52:08.931812 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:08.931769 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rlmqv" podStartSLOduration=9.063345377 podStartE2EDuration="40.931755775s" totalImagesPullingTime="31.868410398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:08.929935402 +0000 UTC m=+41.689006425" watchObservedRunningTime="2026-07-23 21:52:08.931755775 +0000 UTC m=+41.690826785" Jul 23 21:52:10.164942 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.164913 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-42jjx"] Jul 23 21:52:10.187110 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.187077 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x"] Jul 23 21:52:10.187272 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.187154 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.190027 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.190005 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 23 21:52:10.190308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.190288 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 23 21:52:10.190408 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.190353 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.190408 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.190362 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-swb28\"" Jul 23 21:52:10.190523 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.190418 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.196231 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.196214 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 23 21:52:10.207149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.207122 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-42jjx"] Jul 23 21:52:10.207266 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.207157 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x"] Jul 23 21:52:10.207266 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.207247 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.209899 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.209881 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.210214 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.210198 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.210288 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.210221 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 23 21:52:10.210288 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.210245 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qdrww\"" Jul 23 21:52:10.211559 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.211542 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 23 21:52:10.216945 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.216926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59ed4961-d552-4186-8e13-31f800553994-tmp\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.217031 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.216960 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59ed4961-d552-4186-8e13-31f800553994-snapshots\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.217031 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.217163 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217032 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntjh5\" (UniqueName: \"kubernetes.io/projected/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-kube-api-access-ntjh5\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.217163 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217135 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.217250 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217163 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59ed4961-d552-4186-8e13-31f800553994-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.217250 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59ed4961-d552-4186-8e13-31f800553994-serving-cert\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.217317 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217260 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59ed4961-d552-4186-8e13-31f800553994-service-ca-bundle\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.217317 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.217284 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x82gc\" (UniqueName: \"kubernetes.io/projected/59ed4961-d552-4186-8e13-31f800553994-kube-api-access-x82gc\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.266020 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.265988 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk"] Jul 23 21:52:10.285157 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.285126 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6fb9c4db6-5x4fg"] Jul 23 21:52:10.285316 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.285271 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" Jul 23 21:52:10.288574 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.288545 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.288704 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.288585 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-27h2r\"" Jul 23 21:52:10.288894 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.288873 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.303020 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.302997 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk"] Jul 23 21:52:10.303020 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.303034 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fb9c4db6-5x4fg"] Jul 23 21:52:10.303179 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.303129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.306034 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306014 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 23 21:52:10.306248 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306231 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 23 21:52:10.306325 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306304 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 23 21:52:10.306416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306389 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rkgh6\"" Jul 23 21:52:10.306416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306394 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.306539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306433 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.306539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.306504 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 23 21:52:10.317850 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.317829 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59ed4961-d552-4186-8e13-31f800553994-tmp\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.317963 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.317861 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwlws\" (UniqueName: \"kubernetes.io/projected/498767e6-8f98-495f-8d92-82b966ffd20c-kube-api-access-xwlws\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.317963 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.317884 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.317963 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.317943 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-default-certificate\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.317963 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.317954 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:10.318183 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.317969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq4vf\" (UniqueName: \"kubernetes.io/projected/93579de5-b2dd-4ba2-976f-007a2059487b-kube-api-access-lq4vf\") pod \"volume-data-source-validator-78cf4c6f6-x4fzk\" (UID: \"93579de5-b2dd-4ba2-976f-007a2059487b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" Jul 23 21:52:10.318183 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.318003 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls podName:6cd66ba2-7eba-46a5-8f66-c25231d3e0be nodeName:}" failed. No retries permitted until 2026-07-23 21:52:10.817986258 +0000 UTC m=+43.577057249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-phk9x" (UID: "6cd66ba2-7eba-46a5-8f66-c25231d3e0be") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:10.318183 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318144 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.318183 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318175 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59ed4961-d552-4186-8e13-31f800553994-serving-cert\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318205 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59ed4961-d552-4186-8e13-31f800553994-service-ca-bundle\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318233 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x82gc\" (UniqueName: \"kubernetes.io/projected/59ed4961-d552-4186-8e13-31f800553994-kube-api-access-x82gc\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318240 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59ed4961-d552-4186-8e13-31f800553994-tmp\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318264 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59ed4961-d552-4186-8e13-31f800553994-snapshots\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318299 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318330 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.318376 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntjh5\" (UniqueName: \"kubernetes.io/projected/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-kube-api-access-ntjh5\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.318654 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318509 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59ed4961-d552-4186-8e13-31f800553994-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318654 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-stats-auth\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.318828 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318805 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59ed4961-d552-4186-8e13-31f800553994-service-ca-bundle\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318886 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318805 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/59ed4961-d552-4186-8e13-31f800553994-snapshots\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.318922 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.318900 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.319306 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.319289 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59ed4961-d552-4186-8e13-31f800553994-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.321858 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.321836 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/59ed4961-d552-4186-8e13-31f800553994-serving-cert\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.331521 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.331492 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x82gc\" (UniqueName: \"kubernetes.io/projected/59ed4961-d552-4186-8e13-31f800553994-kube-api-access-x82gc\") pod \"insights-operator-555d9d7885-42jjx\" (UID: \"59ed4961-d552-4186-8e13-31f800553994\") " pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.332178 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.332155 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntjh5\" (UniqueName: \"kubernetes.io/projected/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-kube-api-access-ntjh5\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.372490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.372459 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd"] Jul 23 21:52:10.407562 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.407528 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg"] Jul 23 21:52:10.407718 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.407677 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:10.410497 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.410473 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 23 21:52:10.410622 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.410496 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-q7l9j\"" Jul 23 21:52:10.410674 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.410627 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.410726 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.410704 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.419319 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r98jq\" (UniqueName: \"kubernetes.io/projected/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-kube-api-access-r98jq\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:10.419432 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.419432 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.419539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-stats-auth\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.419539 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:10.419539 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419504 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:52:10.419539 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419537 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:10.919515458 +0000 UTC m=+43.678586447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : configmap references non-existent config key: service-ca.crt Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwlws\" (UniqueName: \"kubernetes.io/projected/498767e6-8f98-495f-8d92-82b966ffd20c-kube-api-access-xwlws\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419589 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419595 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-default-certificate\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419603 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765678454c-xhg9v: secret "image-registry-tls" not found Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419628 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:10.9196072 +0000 UTC m=+43.678678201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : secret "router-metrics-certs-default" not found Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419656 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls podName:84b492fd-77e2-4ce2-9a18-5391c94b98d5 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:18.419640961 +0000 UTC m=+51.178711965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls") pod "image-registry-765678454c-xhg9v" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5") : secret "image-registry-tls" not found Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419675 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq4vf\" (UniqueName: \"kubernetes.io/projected/93579de5-b2dd-4ba2-976f-007a2059487b-kube-api-access-lq4vf\") pod \"volume-data-source-validator-78cf4c6f6-x4fzk\" (UID: \"93579de5-b2dd-4ba2-976f-007a2059487b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" Jul 23 21:52:10.419731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:10.420018 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.419746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:10.420018 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419835 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:10.420018 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.419879 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls podName:4de109c8-8b50-46f2-b7ae-e4340a11cdcf nodeName:}" failed. No retries permitted until 2026-07-23 21:52:18.419867288 +0000 UTC m=+51.178938297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls") pod "dns-default-l4gl4" (UID: "4de109c8-8b50-46f2-b7ae-e4340a11cdcf") : secret "dns-default-metrics-tls" not found Jul 23 21:52:10.421450 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.421430 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62"] Jul 23 21:52:10.421570 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.421550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.421902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.421876 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-stats-auth\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.422027 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.421989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-default-certificate\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.425260 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.425241 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 23 21:52:10.425260 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.425251 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-hzdr7\"" Jul 23 21:52:10.425409 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.425244 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 23 21:52:10.425409 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.425302 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.425409 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.425385 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.437781 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.437761 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwlws\" (UniqueName: \"kubernetes.io/projected/498767e6-8f98-495f-8d92-82b966ffd20c-kube-api-access-xwlws\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.439325 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.439306 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq4vf\" (UniqueName: \"kubernetes.io/projected/93579de5-b2dd-4ba2-976f-007a2059487b-kube-api-access-lq4vf\") pod \"volume-data-source-validator-78cf4c6f6-x4fzk\" (UID: \"93579de5-b2dd-4ba2-976f-007a2059487b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" Jul 23 21:52:10.439601 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.439583 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4"] Jul 23 21:52:10.439712 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.439696 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.441843 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.441827 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 23 21:52:10.441924 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.441827 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 23 21:52:10.442425 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.442409 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pq5sg\"" Jul 23 21:52:10.442599 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.442587 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.442886 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.442874 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.455188 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.455168 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd"] Jul 23 21:52:10.455294 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.455201 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4"] Jul 23 21:52:10.455294 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.455214 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62"] Jul 23 21:52:10.455294 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.455255 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg"] Jul 23 21:52:10.455294 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.455292 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" Jul 23 21:52:10.458222 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.458199 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7dm67\"" Jul 23 21:52:10.477998 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.477972 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kmfq9"] Jul 23 21:52:10.493039 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.493013 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.495969 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.495950 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:52:10.496139 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.496123 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 23 21:52:10.496234 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.496218 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9hbdt\"" Jul 23 21:52:10.496325 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.496284 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 23 21:52:10.496389 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.496353 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 23 21:52:10.496475 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.496423 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-42jjx" Jul 23 21:52:10.500210 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.500169 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kmfq9"] Jul 23 21:52:10.506105 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.505260 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 23 21:52:10.520625 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520601 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7v4v\" (UniqueName: \"kubernetes.io/projected/06015c6c-9834-4cc1-9da3-18570c89753d-kube-api-access-k7v4v\") pod \"network-check-source-7bc5b6bf85-7fwz4\" (UID: \"06015c6c-9834-4cc1-9da3-18570c89753d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" Jul 23 21:52:10.520739 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:10.520739 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r98jq\" (UniqueName: \"kubernetes.io/projected/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-kube-api-access-r98jq\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:10.520739 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520714 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc6fs\" (UniqueName: \"kubernetes.io/projected/00df7f87-13d8-4a59-abda-a4f46db66d9a-kube-api-access-mc6fs\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.520869 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af5e4163-1b02-4044-83e1-98a758443469-config\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.520869 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520786 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c0c96d0-bbea-4969-a466-5c46ab6285f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.520869 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.520832 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 21:52:10.521005 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520878 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00df7f87-13d8-4a59-abda-a4f46db66d9a-config\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.521005 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.520916 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls podName:b65f336a-a7f9-4b1a-a4ca-5e15c23871d8 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.020902938 +0000 UTC m=+43.779973926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6vkzd" (UID: "b65f336a-a7f9-4b1a-a4ca-5e15c23871d8") : secret "samples-operator-tls" not found Jul 23 21:52:10.521005 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.520998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af5e4163-1b02-4044-83e1-98a758443469-trusted-ca\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.521192 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.521032 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c0c96d0-bbea-4969-a466-5c46ab6285f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.521192 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.521098 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvhs\" (UniqueName: \"kubernetes.io/projected/af5e4163-1b02-4044-83e1-98a758443469-kube-api-access-njvhs\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.521192 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.521129 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00df7f87-13d8-4a59-abda-a4f46db66d9a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.521192 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.521158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af5e4163-1b02-4044-83e1-98a758443469-serving-cert\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.521341 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.521196 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7hnq\" (UniqueName: \"kubernetes.io/projected/8c0c96d0-bbea-4969-a466-5c46ab6285f8-kube-api-access-t7hnq\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.530741 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.530719 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r98jq\" (UniqueName: \"kubernetes.io/projected/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-kube-api-access-r98jq\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:10.598204 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.598164 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" Jul 23 21:52:10.621724 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621695 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af5e4163-1b02-4044-83e1-98a758443469-config\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.621872 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621757 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c0c96d0-bbea-4969-a466-5c46ab6285f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.621872 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00df7f87-13d8-4a59-abda-a4f46db66d9a-config\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.621872 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621815 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af5e4163-1b02-4044-83e1-98a758443469-trusted-ca\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.621872 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621839 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c0c96d0-bbea-4969-a466-5c46ab6285f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.621872 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621865 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvhs\" (UniqueName: \"kubernetes.io/projected/af5e4163-1b02-4044-83e1-98a758443469-kube-api-access-njvhs\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.622024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621882 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00df7f87-13d8-4a59-abda-a4f46db66d9a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.622024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621898 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af5e4163-1b02-4044-83e1-98a758443469-serving-cert\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.622024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621930 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7hnq\" (UniqueName: \"kubernetes.io/projected/8c0c96d0-bbea-4969-a466-5c46ab6285f8-kube-api-access-t7hnq\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.622024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.621964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7v4v\" (UniqueName: \"kubernetes.io/projected/06015c6c-9834-4cc1-9da3-18570c89753d-kube-api-access-k7v4v\") pod \"network-check-source-7bc5b6bf85-7fwz4\" (UID: \"06015c6c-9834-4cc1-9da3-18570c89753d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" Jul 23 21:52:10.622024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.622019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc6fs\" (UniqueName: \"kubernetes.io/projected/00df7f87-13d8-4a59-abda-a4f46db66d9a-kube-api-access-mc6fs\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.622550 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.622524 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af5e4163-1b02-4044-83e1-98a758443469-config\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.622663 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.622639 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8c0c96d0-bbea-4969-a466-5c46ab6285f8-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.623223 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.623194 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af5e4163-1b02-4044-83e1-98a758443469-trusted-ca\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.623613 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.623592 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00df7f87-13d8-4a59-abda-a4f46db66d9a-config\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.624559 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.624537 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af5e4163-1b02-4044-83e1-98a758443469-serving-cert\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.624934 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.624909 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8c0c96d0-bbea-4969-a466-5c46ab6285f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.625274 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.625250 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00df7f87-13d8-4a59-abda-a4f46db66d9a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.631417 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.631390 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvhs\" (UniqueName: \"kubernetes.io/projected/af5e4163-1b02-4044-83e1-98a758443469-kube-api-access-njvhs\") pod \"console-operator-565b7b569d-kmfq9\" (UID: \"af5e4163-1b02-4044-83e1-98a758443469\") " pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.631646 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.631628 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7hnq\" (UniqueName: \"kubernetes.io/projected/8c0c96d0-bbea-4969-a466-5c46ab6285f8-kube-api-access-t7hnq\") pod \"kube-storage-version-migrator-operator-7658d9b749-6h9vg\" (UID: \"8c0c96d0-bbea-4969-a466-5c46ab6285f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.631961 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.631942 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7v4v\" (UniqueName: \"kubernetes.io/projected/06015c6c-9834-4cc1-9da3-18570c89753d-kube-api-access-k7v4v\") pod \"network-check-source-7bc5b6bf85-7fwz4\" (UID: \"06015c6c-9834-4cc1-9da3-18570c89753d\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" Jul 23 21:52:10.632312 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.632294 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc6fs\" (UniqueName: \"kubernetes.io/projected/00df7f87-13d8-4a59-abda-a4f46db66d9a-kube-api-access-mc6fs\") pod \"service-ca-operator-5bcbc94c78-8xv62\" (UID: \"00df7f87-13d8-4a59-abda-a4f46db66d9a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.661636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.661602 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-42jjx"] Jul 23 21:52:10.664473 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:10.664441 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ed4961_d552_4186_8e13_31f800553994.slice/crio-3e553d80b3bd01ade329281ab5e891ca5d33934c5d45ad4e325052043608a366 WatchSource:0}: Error finding container 3e553d80b3bd01ade329281ab5e891ca5d33934c5d45ad4e325052043608a366: Status 404 returned error can't find the container with id 3e553d80b3bd01ade329281ab5e891ca5d33934c5d45ad4e325052043608a366 Jul 23 21:52:10.736922 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.736893 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" Jul 23 21:52:10.748064 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.748020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" Jul 23 21:52:10.752333 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.752312 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk"] Jul 23 21:52:10.756180 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:10.756154 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93579de5_b2dd_4ba2_976f_007a2059487b.slice/crio-40b2a9a55b5f86fbeee377c9059a86694e2953038d527da58b317048a14a0297 WatchSource:0}: Error finding container 40b2a9a55b5f86fbeee377c9059a86694e2953038d527da58b317048a14a0297: Status 404 returned error can't find the container with id 40b2a9a55b5f86fbeee377c9059a86694e2953038d527da58b317048a14a0297 Jul 23 21:52:10.763141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.763121 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" Jul 23 21:52:10.803455 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.803425 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:10.823581 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.823556 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:10.823807 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.823722 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:10.823807 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.823792 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls podName:6cd66ba2-7eba-46a5-8f66-c25231d3e0be nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.823772789 +0000 UTC m=+44.582843782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-phk9x" (UID: "6cd66ba2-7eba-46a5-8f66-c25231d3e0be") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:10.902670 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.902560 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg"] Jul 23 21:52:10.905224 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:10.905197 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0c96d0_bbea_4969_a466_5c46ab6285f8.slice/crio-8c109fca62d8158c0a4f2c950aea4d0360d2e96f705f36f198ca35861d6192d3 WatchSource:0}: Error finding container 8c109fca62d8158c0a4f2c950aea4d0360d2e96f705f36f198ca35861d6192d3: Status 404 returned error can't find the container with id 8c109fca62d8158c0a4f2c950aea4d0360d2e96f705f36f198ca35861d6192d3 Jul 23 21:52:10.914362 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.914287 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" event={"ID":"8c0c96d0-bbea-4969-a466-5c46ab6285f8","Type":"ContainerStarted","Data":"8c109fca62d8158c0a4f2c950aea4d0360d2e96f705f36f198ca35861d6192d3"} Jul 23 21:52:10.915649 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.915587 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-42jjx" event={"ID":"59ed4961-d552-4186-8e13-31f800553994","Type":"ContainerStarted","Data":"3e553d80b3bd01ade329281ab5e891ca5d33934c5d45ad4e325052043608a366"} Jul 23 21:52:10.917162 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.917046 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" event={"ID":"93579de5-b2dd-4ba2-976f-007a2059487b","Type":"ContainerStarted","Data":"40b2a9a55b5f86fbeee377c9059a86694e2953038d527da58b317048a14a0297"} Jul 23 21:52:10.926920 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.925489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.926920 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.925533 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:10.926920 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.925665 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:52:10.926920 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.925725 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.92570596 +0000 UTC m=+44.684776962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : secret "router-metrics-certs-default" not found Jul 23 21:52:10.926920 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:10.925802 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.925791683 +0000 UTC m=+44.684862686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : configmap references non-existent config key: service-ca.crt Jul 23 21:52:10.955001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.954971 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62"] Jul 23 21:52:10.974600 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.974576 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4"] Jul 23 21:52:10.977664 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:10.977631 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06015c6c_9834_4cc1_9da3_18570c89753d.slice/crio-971e303fed84ee4338900293e52da3bc7bdd8b0199b3ec92c72df8f13686f9aa WatchSource:0}: Error finding container 971e303fed84ee4338900293e52da3bc7bdd8b0199b3ec92c72df8f13686f9aa: Status 404 returned error can't find the container with id 971e303fed84ee4338900293e52da3bc7bdd8b0199b3ec92c72df8f13686f9aa Jul 23 21:52:10.998197 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:10.998171 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-kmfq9"] Jul 23 21:52:11.000872 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:11.000845 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5e4163_1b02_4044_83e1_98a758443469.slice/crio-ca4af3f7c42727e6fb88272680e5a9e8ca5992510775c448b3deaa03eab7bf4b WatchSource:0}: Error finding container ca4af3f7c42727e6fb88272680e5a9e8ca5992510775c448b3deaa03eab7bf4b: Status 404 returned error can't find the container with id ca4af3f7c42727e6fb88272680e5a9e8ca5992510775c448b3deaa03eab7bf4b Jul 23 21:52:11.026555 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.026529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:11.026693 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.026676 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 21:52:11.026753 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.026744 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls podName:b65f336a-a7f9-4b1a-a4ca-5e15c23871d8 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:12.026729054 +0000 UTC m=+44.785800043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6vkzd" (UID: "b65f336a-a7f9-4b1a-a4ca-5e15c23871d8") : secret "samples-operator-tls" not found Jul 23 21:52:11.835645 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.835605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:11.836123 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.835897 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:11.836123 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.835963 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls podName:6cd66ba2-7eba-46a5-8f66-c25231d3e0be nodeName:}" failed. No retries permitted until 2026-07-23 21:52:13.835945303 +0000 UTC m=+46.595016298 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-phk9x" (UID: "6cd66ba2-7eba-46a5-8f66-c25231d3e0be") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:11.922605 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.922567 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" event={"ID":"06015c6c-9834-4cc1-9da3-18570c89753d","Type":"ContainerStarted","Data":"971e303fed84ee4338900293e52da3bc7bdd8b0199b3ec92c72df8f13686f9aa"} Jul 23 21:52:11.924407 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.924378 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" event={"ID":"00df7f87-13d8-4a59-abda-a4f46db66d9a","Type":"ContainerStarted","Data":"40e7aab99344d15a0c906304f754547954f8f73958fb7506321b35cac72d4532"} Jul 23 21:52:11.928027 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.927974 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" event={"ID":"af5e4163-1b02-4044-83e1-98a758443469","Type":"ContainerStarted","Data":"ca4af3f7c42727e6fb88272680e5a9e8ca5992510775c448b3deaa03eab7bf4b"} Jul 23 21:52:11.936649 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.936607 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:11.936649 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:11.936634 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:11.936861 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.936818 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:52:11.936913 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.936875 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:13.936858159 +0000 UTC m=+46.695929147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : secret "router-metrics-certs-default" not found Jul 23 21:52:11.936913 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:11.936890 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:13.936882536 +0000 UTC m=+46.695953526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : configmap references non-existent config key: service-ca.crt Jul 23 21:52:12.037906 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:12.037803 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:12.038123 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:12.038004 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 21:52:12.038123 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:12.038094 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls podName:b65f336a-a7f9-4b1a-a4ca-5e15c23871d8 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:14.038048737 +0000 UTC m=+46.797119739 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6vkzd" (UID: "b65f336a-a7f9-4b1a-a4ca-5e15c23871d8") : secret "samples-operator-tls" not found Jul 23 21:52:13.857527 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:13.857488 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:13.857997 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:13.857643 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:13.857997 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:13.857709 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls podName:6cd66ba2-7eba-46a5-8f66-c25231d3e0be nodeName:}" failed. No retries permitted until 2026-07-23 21:52:17.857690942 +0000 UTC m=+50.616761934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-phk9x" (UID: "6cd66ba2-7eba-46a5-8f66-c25231d3e0be") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:13.958808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:13.958775 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:13.958808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:13.958814 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:13.958987 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:13.958944 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:17.958924555 +0000 UTC m=+50.717995558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : configmap references non-existent config key: service-ca.crt Jul 23 21:52:13.958987 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:13.958954 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:52:13.959079 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:13.958990 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:17.958980209 +0000 UTC m=+50.718051216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : secret "router-metrics-certs-default" not found Jul 23 21:52:14.060469 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:14.060433 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:14.060621 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:14.060440 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 21:52:14.060621 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:14.060579 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls podName:b65f336a-a7f9-4b1a-a4ca-5e15c23871d8 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:18.060563598 +0000 UTC m=+50.819634587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6vkzd" (UID: "b65f336a-a7f9-4b1a-a4ca-5e15c23871d8") : secret "samples-operator-tls" not found Jul 23 21:52:17.897656 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.897610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:17.898164 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:17.897759 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:17.898164 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:17.897825 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls podName:6cd66ba2-7eba-46a5-8f66-c25231d3e0be nodeName:}" failed. No retries permitted until 2026-07-23 21:52:25.897806162 +0000 UTC m=+58.656877157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-phk9x" (UID: "6cd66ba2-7eba-46a5-8f66-c25231d3e0be") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:17.947931 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.947893 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-42jjx" event={"ID":"59ed4961-d552-4186-8e13-31f800553994","Type":"ContainerStarted","Data":"51bc84e656059c36683ed98ff53f623045fcb7c1309a98ecb1104e9b902434d5"} Jul 23 21:52:17.949506 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.949478 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" event={"ID":"93579de5-b2dd-4ba2-976f-007a2059487b","Type":"ContainerStarted","Data":"0fc8e26739c2ebd62d1b1088ff950e9f3ad7cbe631803a81fd73b174565e9ab7"} Jul 23 21:52:17.951184 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.951165 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/0.log" Jul 23 21:52:17.951288 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.951199 2579 generic.go:356] "Generic (PLEG): container finished" podID="af5e4163-1b02-4044-83e1-98a758443469" containerID="436aa123b6f54076a4f41c669a38ae32660982ae6ecf12b85cc1cde64e0b29f7" exitCode=255 Jul 23 21:52:17.951355 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.951299 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" event={"ID":"af5e4163-1b02-4044-83e1-98a758443469","Type":"ContainerDied","Data":"436aa123b6f54076a4f41c669a38ae32660982ae6ecf12b85cc1cde64e0b29f7"} Jul 23 21:52:17.951629 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.951601 2579 scope.go:117] "RemoveContainer" containerID="436aa123b6f54076a4f41c669a38ae32660982ae6ecf12b85cc1cde64e0b29f7" Jul 23 21:52:17.953171 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.953148 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" event={"ID":"8c0c96d0-bbea-4969-a466-5c46ab6285f8","Type":"ContainerStarted","Data":"ab0d60d0b5e70123704d0646210d913492e105c1ad2a1b282086dc111ef8765a"} Jul 23 21:52:17.954830 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.954804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" event={"ID":"06015c6c-9834-4cc1-9da3-18570c89753d","Type":"ContainerStarted","Data":"a378660886add573094465c28277833ed8df719b9aeae7a99a2a6102662a458a"} Jul 23 21:52:17.956258 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.956240 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" event={"ID":"00df7f87-13d8-4a59-abda-a4f46db66d9a","Type":"ContainerStarted","Data":"6fd8f0f4bc944ddea8e67cba9fb7328cc78770caa069e3829888a377fe105f93"} Jul 23 21:52:17.966556 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.966512 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-42jjx" podStartSLOduration=1.550473521 podStartE2EDuration="7.966500155s" totalImagesPullingTime="6.416026634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:17.965689594 +0000 UTC m=+50.724760641" watchObservedRunningTime="2026-07-23 21:52:17.966500155 +0000 UTC m=+50.725571179" Jul 23 21:52:17.988329 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.988280 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-x4fzk" podStartSLOduration=1.734267859 podStartE2EDuration="7.988259912s" totalImagesPullingTime="6.253992053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:17.98781598 +0000 UTC m=+50.746886993" watchObservedRunningTime="2026-07-23 21:52:17.988259912 +0000 UTC m=+50.747330924" Jul 23 21:52:17.998655 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.998568 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:17.998655 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:17.998610 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:18.000127 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:17.999144 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:25.999126295 +0000 UTC m=+58.758197291 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : configmap references non-existent config key: service-ca.crt Jul 23 21:52:18.000127 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:17.999228 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:52:18.000127 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:17.999267 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:25.999253432 +0000 UTC m=+58.758324437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : secret "router-metrics-certs-default" not found Jul 23 21:52:18.010635 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.010588 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" podStartSLOduration=1.835853263 podStartE2EDuration="8.010568437s" totalImagesPullingTime="6.174715174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:18.009981813 +0000 UTC m=+50.769052823" watchObservedRunningTime="2026-07-23 21:52:18.010568437 +0000 UTC m=+50.769639451" Jul 23 21:52:18.031677 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.031587 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-7fwz4" podStartSLOduration=1.918946376 podStartE2EDuration="8.03156883s" totalImagesPullingTime="6.112622454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:18.03136105 +0000 UTC m=+50.790432061" watchObservedRunningTime="2026-07-23 21:52:18.03156883 +0000 UTC m=+50.790639840" Jul 23 21:52:18.076561 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.076504 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" podStartSLOduration=1.962338527 podStartE2EDuration="8.076482079s" totalImagesPullingTime="6.114143552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:18.074429692 +0000 UTC m=+50.833500704" watchObservedRunningTime="2026-07-23 21:52:18.076482079 +0000 UTC m=+50.835553093" Jul 23 21:52:18.099529 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.099416 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:18.099691 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.099595 2579 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 23 21:52:18.099776 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.099683 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls podName:b65f336a-a7f9-4b1a-a4ca-5e15c23871d8 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:26.099663644 +0000 UTC m=+58.858734645 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-6vkzd" (UID: "b65f336a-a7f9-4b1a-a4ca-5e15c23871d8") : secret "samples-operator-tls" not found Jul 23 21:52:18.419068 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.417686 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5fhxg"] Jul 23 21:52:18.435558 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.435525 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.437769 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.437738 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5fhxg"] Jul 23 21:52:18.438822 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.438655 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 21:52:18.438822 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.438699 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jtqd5\"" Jul 23 21:52:18.438822 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.438708 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 21:52:18.502765 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.502721 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.502957 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.502775 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eaafa37-b84a-489c-9d61-75852a35147a-data-volume\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.502957 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.502875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:18.503102 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.503003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5eaafa37-b84a-489c-9d61-75852a35147a-crio-socket\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.503102 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.503040 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:18.503102 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.503044 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.503251 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.503136 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls podName:4de109c8-8b50-46f2-b7ae-e4340a11cdcf nodeName:}" failed. No retries permitted until 2026-07-23 21:52:34.503105482 +0000 UTC m=+67.262176475 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls") pod "dns-default-l4gl4" (UID: "4de109c8-8b50-46f2-b7ae-e4340a11cdcf") : secret "dns-default-metrics-tls" not found Jul 23 21:52:18.503251 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.503186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgmst\" (UniqueName: \"kubernetes.io/projected/5eaafa37-b84a-489c-9d61-75852a35147a-kube-api-access-lgmst\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.503251 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.503224 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:18.503400 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.503338 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:18.503400 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.503351 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-765678454c-xhg9v: secret "image-registry-tls" not found Jul 23 21:52:18.503400 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.503396 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls podName:84b492fd-77e2-4ce2-9a18-5391c94b98d5 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:34.503381057 +0000 UTC m=+67.262452049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls") pod "image-registry-765678454c-xhg9v" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5") : secret "image-registry-tls" not found Jul 23 21:52:18.603898 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.603864 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5eaafa37-b84a-489c-9d61-75852a35147a-crio-socket\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.604093 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.603906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.604093 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.603934 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgmst\" (UniqueName: \"kubernetes.io/projected/5eaafa37-b84a-489c-9d61-75852a35147a-kube-api-access-lgmst\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.604093 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.604003 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:18.604093 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.604010 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5eaafa37-b84a-489c-9d61-75852a35147a-crio-socket\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.604093 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.604005 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.604093 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.604084 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:19.104049542 +0000 UTC m=+51.863120551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : configmap "kube-rbac-proxy" not found Jul 23 21:52:18.604413 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.604102 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:18.604413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.604132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eaafa37-b84a-489c-9d61-75852a35147a-data-volume\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.604413 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.604139 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:19.104127954 +0000 UTC m=+51.863198945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:18.604413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.604403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5eaafa37-b84a-489c-9d61-75852a35147a-data-volume\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.615860 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.615834 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgmst\" (UniqueName: \"kubernetes.io/projected/5eaafa37-b84a-489c-9d61-75852a35147a-kube-api-access-lgmst\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:18.960129 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.960102 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 21:52:18.960516 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.960491 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/0.log" Jul 23 21:52:18.960574 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.960522 2579 generic.go:356] "Generic (PLEG): container finished" podID="af5e4163-1b02-4044-83e1-98a758443469" containerID="78cbe3fb7304cd7294e335bcbffdc6696ff9ddc53947ef622d4f7c50be0761f1" exitCode=255 Jul 23 21:52:18.960658 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.960624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" event={"ID":"af5e4163-1b02-4044-83e1-98a758443469","Type":"ContainerDied","Data":"78cbe3fb7304cd7294e335bcbffdc6696ff9ddc53947ef622d4f7c50be0761f1"} Jul 23 21:52:18.960735 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.960723 2579 scope.go:117] "RemoveContainer" containerID="436aa123b6f54076a4f41c669a38ae32660982ae6ecf12b85cc1cde64e0b29f7" Jul 23 21:52:18.960953 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:18.960936 2579 scope.go:117] "RemoveContainer" containerID="78cbe3fb7304cd7294e335bcbffdc6696ff9ddc53947ef622d4f7c50be0761f1" Jul 23 21:52:18.961142 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:18.961122 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kmfq9_openshift-console-operator(af5e4163-1b02-4044-83e1-98a758443469)\"" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" podUID="af5e4163-1b02-4044-83e1-98a758443469" Jul 23 21:52:19.110151 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:19.110115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:19.110151 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:19.110132 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:19.110371 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:19.110198 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:20.110183938 +0000 UTC m=+52.869254930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : configmap "kube-rbac-proxy" not found Jul 23 21:52:19.110371 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:19.110226 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:19.110371 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:19.110335 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:19.110371 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:19.110363 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:20.110356267 +0000 UTC m=+52.869427256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:19.964984 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:19.964959 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 21:52:19.965379 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:19.965314 2579 scope.go:117] "RemoveContainer" containerID="78cbe3fb7304cd7294e335bcbffdc6696ff9ddc53947ef622d4f7c50be0761f1" Jul 23 21:52:19.965474 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:19.965456 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kmfq9_openshift-console-operator(af5e4163-1b02-4044-83e1-98a758443469)\"" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" podUID="af5e4163-1b02-4044-83e1-98a758443469" Jul 23 21:52:20.030756 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:20.030729 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rqrtl_5af88a0e-a538-435c-a540-65e938e7ecf6/dns-node-resolver/0.log" Jul 23 21:52:20.119624 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:20.119584 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:20.119794 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:20.119649 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:20.119794 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:20.119712 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:20.119794 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:20.119751 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:20.119794 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:20.119784 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:22.119763043 +0000 UTC m=+54.878834032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : configmap "kube-rbac-proxy" not found Jul 23 21:52:20.119931 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:20.119803 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:22.119796142 +0000 UTC m=+54.878867130 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:20.804141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:20.804106 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:20.804428 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:20.804167 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:20.968204 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:20.968169 2579 scope.go:117] "RemoveContainer" containerID="78cbe3fb7304cd7294e335bcbffdc6696ff9ddc53947ef622d4f7c50be0761f1" Jul 23 21:52:20.968641 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:20.968371 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-kmfq9_openshift-console-operator(af5e4163-1b02-4044-83e1-98a758443469)\"" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" podUID="af5e4163-1b02-4044-83e1-98a758443469" Jul 23 21:52:21.235284 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:21.235252 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w4lz2_7c95a45f-e8ec-402b-97eb-9089ed4c2bce/node-ca/0.log" Jul 23 21:52:21.835389 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:21.835365 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-966bw_baf45753-a888-4811-b238-767035037d2e/serve-healthcheck-canary/0.log" Jul 23 21:52:22.136832 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:22.136795 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:22.137225 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:22.136911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:22.137225 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:22.136954 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:22.137225 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:22.136981 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:22.137225 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:22.137027 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:26.137012045 +0000 UTC m=+58.896083035 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : configmap "kube-rbac-proxy" not found Jul 23 21:52:22.137225 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:22.137038 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:26.137032764 +0000 UTC m=+58.896103752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:25.971968 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:25.971927 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:25.972430 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:25.972110 2579 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:25.972430 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:25.972178 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls podName:6cd66ba2-7eba-46a5-8f66-c25231d3e0be nodeName:}" failed. No retries permitted until 2026-07-23 21:52:41.972160558 +0000 UTC m=+74.731231551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-phk9x" (UID: "6cd66ba2-7eba-46a5-8f66-c25231d3e0be") : secret "cluster-monitoring-operator-tls" not found Jul 23 21:52:26.072690 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.072644 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:26.072690 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.072691 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:26.072887 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.072833 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:42.072814564 +0000 UTC m=+74.831885552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : configmap references non-existent config key: service-ca.crt Jul 23 21:52:26.072887 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.072868 2579 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 23 21:52:26.072957 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.072930 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs podName:498767e6-8f98-495f-8d92-82b966ffd20c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:42.072915276 +0000 UTC m=+74.831986284 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs") pod "router-default-6fb9c4db6-5x4fg" (UID: "498767e6-8f98-495f-8d92-82b966ffd20c") : secret "router-metrics-certs-default" not found Jul 23 21:52:26.173500 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.173465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:26.173635 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.173517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:26.173635 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.173609 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:26.173710 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.173675 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:34.173658642 +0000 UTC m=+66.932729651 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:26.173777 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.173745 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:26.173828 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.173809 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:26.173876 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:26.173841 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:34.173832262 +0000 UTC m=+66.932903251 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : configmap "kube-rbac-proxy" not found Jul 23 21:52:26.175924 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.175891 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b65f336a-a7f9-4b1a-a4ca-5e15c23871d8-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-6vkzd\" (UID: \"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:26.316521 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.316430 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" Jul 23 21:52:26.443227 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.443098 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd"] Jul 23 21:52:26.986956 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:26.986915 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" event={"ID":"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8","Type":"ContainerStarted","Data":"14b37ea9c313ac2b39c6cb5aeb706089c9a9ae676c8c4cdfb2c62f80bf335afb"} Jul 23 21:52:28.880612 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:28.880586 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlb2q" Jul 23 21:52:28.995270 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:28.995189 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" event={"ID":"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8","Type":"ContainerStarted","Data":"9dacf8b21e56d1754a8494523c3c72e16353a3d4570a83ffb716ba157d574d1c"} Jul 23 21:52:28.995401 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:28.995273 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" event={"ID":"b65f336a-a7f9-4b1a-a4ca-5e15c23871d8","Type":"ContainerStarted","Data":"ca5b98d90ce3b62f15d43244addf5f67eafd951af62e875a019525f4249ac6f4"} Jul 23 21:52:29.014116 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:29.014038 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-6vkzd" podStartSLOduration=16.761422029 podStartE2EDuration="19.01402292s" totalImagesPullingTime="2.252600891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:29.013164128 +0000 UTC m=+61.772235196" watchObservedRunningTime="2026-07-23 21:52:29.01402292 +0000 UTC m=+61.773093930" Jul 23 21:52:33.540342 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.540302 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:33.540805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.540360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:52:33.540805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.540557 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:52:33.542780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.542759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wd2m\" (UniqueName: \"kubernetes.io/projected/c25b748b-2ddd-40fc-9127-4c8cdd8ac42e-kube-api-access-6wd2m\") pod \"network-check-target-kncqt\" (UID: \"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e\") " pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:33.543122 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.543102 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 21:52:33.543238 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.543221 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 21:52:33.552569 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.552549 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac4ced1f-d98f-4f63-94bd-8dd118c70c3e-original-pull-secret\") pod \"global-pull-secret-syncer-mgtn9\" (UID: \"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e\") " pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:52:33.552672 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.552621 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21a14840-5dfc-4c8d-91b1-ef5db29807f6-metrics-certs\") pod \"network-metrics-daemon-gp9l9\" (UID: \"21a14840-5dfc-4c8d-91b1-ef5db29807f6\") " pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:52:33.688963 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.688936 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxtft\"" Jul 23 21:52:33.694528 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.694505 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mgtn9" Jul 23 21:52:33.696392 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.696372 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-gp9l9" Jul 23 21:52:33.704645 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.704616 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9zwh\"" Jul 23 21:52:33.712445 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.712422 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:33.845408 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.845386 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-gp9l9"] Jul 23 21:52:33.848006 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:33.847966 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21a14840_5dfc_4c8d_91b1_ef5db29807f6.slice/crio-7b4157f19e2060a8b30062895610c62f693eb9da4016fab7e3028ee18a8ae61a WatchSource:0}: Error finding container 7b4157f19e2060a8b30062895610c62f693eb9da4016fab7e3028ee18a8ae61a: Status 404 returned error can't find the container with id 7b4157f19e2060a8b30062895610c62f693eb9da4016fab7e3028ee18a8ae61a Jul 23 21:52:33.859040 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.859012 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mgtn9"] Jul 23 21:52:33.861637 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:33.861615 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4ced1f_d98f_4f63_94bd_8dd118c70c3e.slice/crio-5551e7316ec87009252d9b3e10e7cdd45265a83a140dd08e4107c732c69e4562 WatchSource:0}: Error finding container 5551e7316ec87009252d9b3e10e7cdd45265a83a140dd08e4107c732c69e4562: Status 404 returned error can't find the container with id 5551e7316ec87009252d9b3e10e7cdd45265a83a140dd08e4107c732c69e4562 Jul 23 21:52:33.876823 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:33.876802 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kncqt"] Jul 23 21:52:33.878998 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:33.878950 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25b748b_2ddd_40fc_9127_4c8cdd8ac42e.slice/crio-851091eea29e5d46b51651a53c40e4c08d1ba8e7aec85c673fc98bf56d842978 WatchSource:0}: Error finding container 851091eea29e5d46b51651a53c40e4c08d1ba8e7aec85c673fc98bf56d842978: Status 404 returned error can't find the container with id 851091eea29e5d46b51651a53c40e4c08d1ba8e7aec85c673fc98bf56d842978 Jul 23 21:52:34.010674 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.010638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gp9l9" event={"ID":"21a14840-5dfc-4c8d-91b1-ef5db29807f6","Type":"ContainerStarted","Data":"7b4157f19e2060a8b30062895610c62f693eb9da4016fab7e3028ee18a8ae61a"} Jul 23 21:52:34.011782 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.011761 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mgtn9" event={"ID":"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e","Type":"ContainerStarted","Data":"5551e7316ec87009252d9b3e10e7cdd45265a83a140dd08e4107c732c69e4562"} Jul 23 21:52:34.012990 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.012966 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kncqt" event={"ID":"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e","Type":"ContainerStarted","Data":"ffc3256124bfce48f6b5ac3f4343908daddca284a133af7d62cffa331b662c12"} Jul 23 21:52:34.013117 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.012998 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kncqt" event={"ID":"c25b748b-2ddd-40fc-9127-4c8cdd8ac42e","Type":"ContainerStarted","Data":"851091eea29e5d46b51651a53c40e4c08d1ba8e7aec85c673fc98bf56d842978"} Jul 23 21:52:34.013117 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.013100 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:52:34.247980 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.247940 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:34.248173 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.248041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:34.248173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:34.248110 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:34.248306 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:34.248176 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls podName:5eaafa37-b84a-489c-9d61-75852a35147a nodeName:}" failed. No retries permitted until 2026-07-23 21:52:50.248160974 +0000 UTC m=+83.007231963 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5fhxg" (UID: "5eaafa37-b84a-489c-9d61-75852a35147a") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:34.248626 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.248601 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5eaafa37-b84a-489c-9d61-75852a35147a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:34.550648 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.550576 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:34.551047 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.550674 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:34.553366 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.553335 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4de109c8-8b50-46f2-b7ae-e4340a11cdcf-metrics-tls\") pod \"dns-default-l4gl4\" (UID: \"4de109c8-8b50-46f2-b7ae-e4340a11cdcf\") " pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:34.553784 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.553759 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"image-registry-765678454c-xhg9v\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:34.703192 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.703157 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hkh5w\"" Jul 23 21:52:34.711432 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.711403 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:34.722495 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.722473 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq952\"" Jul 23 21:52:34.729805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.729786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:34.973633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.973406 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kncqt" podStartSLOduration=67.973382418 podStartE2EDuration="1m7.973382418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:34.031964806 +0000 UTC m=+66.791035818" watchObservedRunningTime="2026-07-23 21:52:34.973382418 +0000 UTC m=+67.732453431" Jul 23 21:52:34.975133 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.975100 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-765678454c-xhg9v"] Jul 23 21:52:34.994722 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:34.994696 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l4gl4"] Jul 23 21:52:35.037816 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:35.037778 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84b492fd_77e2_4ce2_9a18_5391c94b98d5.slice/crio-34cffc13e01641613c80e6e98123098ccdea043cd8884a5d771536ff9302fb50 WatchSource:0}: Error finding container 34cffc13e01641613c80e6e98123098ccdea043cd8884a5d771536ff9302fb50: Status 404 returned error can't find the container with id 34cffc13e01641613c80e6e98123098ccdea043cd8884a5d771536ff9302fb50 Jul 23 21:52:35.038970 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:35.038938 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de109c8_8b50_46f2_b7ae_e4340a11cdcf.slice/crio-3c25caf660631014665e07c21cf88317f0c1f674cea38f7d6be7340e697b932e WatchSource:0}: Error finding container 3c25caf660631014665e07c21cf88317f0c1f674cea38f7d6be7340e697b932e: Status 404 returned error can't find the container with id 3c25caf660631014665e07c21cf88317f0c1f674cea38f7d6be7340e697b932e Jul 23 21:52:36.023320 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.023285 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gp9l9" event={"ID":"21a14840-5dfc-4c8d-91b1-ef5db29807f6","Type":"ContainerStarted","Data":"acfe43b7205d48c12e4905a751c02d3fe7cef815444778c6eff7fa3d0fe10c4b"} Jul 23 21:52:36.023776 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.023351 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-gp9l9" event={"ID":"21a14840-5dfc-4c8d-91b1-ef5db29807f6","Type":"ContainerStarted","Data":"e2faf0391fa2e2707a7552148693ce86ee9eef40d4e8c403bfa8e11abf1bdac4"} Jul 23 21:52:36.024737 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.024708 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l4gl4" event={"ID":"4de109c8-8b50-46f2-b7ae-e4340a11cdcf","Type":"ContainerStarted","Data":"3c25caf660631014665e07c21cf88317f0c1f674cea38f7d6be7340e697b932e"} Jul 23 21:52:36.026572 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.026487 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765678454c-xhg9v" event={"ID":"84b492fd-77e2-4ce2-9a18-5391c94b98d5","Type":"ContainerStarted","Data":"0e768e36483b0913512c3901c71e3748ab3c3b213bf7f2593fda8dd33071aa07"} Jul 23 21:52:36.026572 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.026529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765678454c-xhg9v" event={"ID":"84b492fd-77e2-4ce2-9a18-5391c94b98d5","Type":"ContainerStarted","Data":"34cffc13e01641613c80e6e98123098ccdea043cd8884a5d771536ff9302fb50"} Jul 23 21:52:36.026739 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.026648 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:36.043046 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.042615 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-gp9l9" podStartSLOduration=66.823939858 podStartE2EDuration="1m8.04256518s" totalImagesPullingTime="1.218625322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:36.041534664 +0000 UTC m=+68.800605667" watchObservedRunningTime="2026-07-23 21:52:36.04256518 +0000 UTC m=+68.801636194" Jul 23 21:52:36.064965 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.064387 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-765678454c-xhg9v" podStartSLOduration=45.064371438 podStartE2EDuration="45.064371438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:36.061476028 +0000 UTC m=+68.820547093" watchObservedRunningTime="2026-07-23 21:52:36.064371438 +0000 UTC m=+68.823442449" Jul 23 21:52:36.673152 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:36.673120 2579 scope.go:117] "RemoveContainer" containerID="78cbe3fb7304cd7294e335bcbffdc6696ff9ddc53947ef622d4f7c50be0761f1" Jul 23 21:52:39.045102 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.045042 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mgtn9" event={"ID":"ac4ced1f-d98f-4f63-94bd-8dd118c70c3e","Type":"ContainerStarted","Data":"4b66f155132710a13039d00e75e928a9d447509392765e7bbebb8c667b906570"} Jul 23 21:52:39.047071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.047028 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l4gl4" event={"ID":"4de109c8-8b50-46f2-b7ae-e4340a11cdcf","Type":"ContainerStarted","Data":"f06d8d36f1321875f9e1e7a193fa403bc536633592149d03829f46e2a47c4da7"} Jul 23 21:52:39.047208 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.047108 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l4gl4" event={"ID":"4de109c8-8b50-46f2-b7ae-e4340a11cdcf","Type":"ContainerStarted","Data":"5671b3ba401ef092ef8b47ef072b7e5cf6174d76e828c98b98ce317e355e18b8"} Jul 23 21:52:39.047208 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.047140 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:39.049018 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.048998 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 21:52:39.049155 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.049072 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" event={"ID":"af5e4163-1b02-4044-83e1-98a758443469","Type":"ContainerStarted","Data":"666480fecd273952fc2c3903cc4d8254cb907366defa07060a47df0318f6ba73"} Jul 23 21:52:39.049339 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.049318 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:39.055194 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.055174 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" Jul 23 21:52:39.064816 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.064750 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mgtn9" podStartSLOduration=66.71513171 podStartE2EDuration="1m11.064735986s" totalImagesPullingTime="4.349604276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:39.064070477 +0000 UTC m=+71.823141486" watchObservedRunningTime="2026-07-23 21:52:39.064735986 +0000 UTC m=+71.823806997" Jul 23 21:52:39.084008 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.083958 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-kmfq9" podStartSLOduration=23.003494819 podStartE2EDuration="29.083945368s" totalImagesPullingTime="6.080450549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:39.083765231 +0000 UTC m=+71.842836241" watchObservedRunningTime="2026-07-23 21:52:39.083945368 +0000 UTC m=+71.843016379" Jul 23 21:52:39.102974 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:39.102933 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l4gl4" podStartSLOduration=33.967125312 podStartE2EDuration="37.10291977s" totalImagesPullingTime="3.135794458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:39.102438031 +0000 UTC m=+71.861509044" watchObservedRunningTime="2026-07-23 21:52:39.10291977 +0000 UTC m=+71.861990817" Jul 23 21:52:42.011990 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.011946 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:42.014357 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.014334 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6cd66ba2-7eba-46a5-8f66-c25231d3e0be-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-phk9x\" (UID: \"6cd66ba2-7eba-46a5-8f66-c25231d3e0be\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:42.017705 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.017684 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qdrww\"" Jul 23 21:52:42.026065 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.026038 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" Jul 23 21:52:42.112611 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.112586 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:42.112743 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.112618 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:42.113383 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.113354 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/498767e6-8f98-495f-8d92-82b966ffd20c-service-ca-bundle\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:42.114818 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.114795 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/498767e6-8f98-495f-8d92-82b966ffd20c-metrics-certs\") pod \"router-default-6fb9c4db6-5x4fg\" (UID: \"498767e6-8f98-495f-8d92-82b966ffd20c\") " pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:42.153664 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.153637 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x"] Jul 23 21:52:42.157272 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:42.157244 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd66ba2_7eba_46a5_8f66_c25231d3e0be.slice/crio-7ca165ee121e7e21a1ec6d28431b6a5d6b2fee269059e864f342d6c44a845acc WatchSource:0}: Error finding container 7ca165ee121e7e21a1ec6d28431b6a5d6b2fee269059e864f342d6c44a845acc: Status 404 returned error can't find the container with id 7ca165ee121e7e21a1ec6d28431b6a5d6b2fee269059e864f342d6c44a845acc Jul 23 21:52:42.415113 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.415085 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rkgh6\"" Jul 23 21:52:42.423450 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.423426 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:42.554000 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:42.553973 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fb9c4db6-5x4fg"] Jul 23 21:52:42.556428 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:42.556397 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod498767e6_8f98_495f_8d92_82b966ffd20c.slice/crio-d258ec1c2db00f56e58e0e311fe4d0ce2b6542c6c250bc93d5737232868d4f95 WatchSource:0}: Error finding container d258ec1c2db00f56e58e0e311fe4d0ce2b6542c6c250bc93d5737232868d4f95: Status 404 returned error can't find the container with id d258ec1c2db00f56e58e0e311fe4d0ce2b6542c6c250bc93d5737232868d4f95 Jul 23 21:52:43.067035 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:43.066997 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" event={"ID":"498767e6-8f98-495f-8d92-82b966ffd20c","Type":"ContainerStarted","Data":"26003cfd0c4315797ca03bebb10e9e4bd763fdd06f9abb146ebb021c2d75c23a"} Jul 23 21:52:43.067502 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:43.067071 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" event={"ID":"498767e6-8f98-495f-8d92-82b966ffd20c","Type":"ContainerStarted","Data":"d258ec1c2db00f56e58e0e311fe4d0ce2b6542c6c250bc93d5737232868d4f95"} Jul 23 21:52:43.068285 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:43.068258 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" event={"ID":"6cd66ba2-7eba-46a5-8f66-c25231d3e0be","Type":"ContainerStarted","Data":"7ca165ee121e7e21a1ec6d28431b6a5d6b2fee269059e864f342d6c44a845acc"} Jul 23 21:52:43.090413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:43.090361 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" podStartSLOduration=33.090341828 podStartE2EDuration="33.090341828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:43.088466191 +0000 UTC m=+75.847537201" watchObservedRunningTime="2026-07-23 21:52:43.090341828 +0000 UTC m=+75.849412841" Jul 23 21:52:43.423928 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:43.423872 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:43.427882 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:43.427860 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:44.071833 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.071808 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:44.073143 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.073124 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6fb9c4db6-5x4fg" Jul 23 21:52:44.691088 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.691040 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s"] Jul 23 21:52:44.715153 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.715114 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s"] Jul 23 21:52:44.715308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.715212 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:44.718336 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.718314 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6lqrh\"" Jul 23 21:52:44.718481 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.718461 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 23 21:52:44.739090 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.739066 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96444638-fc69-42cc-ad26-d99bc1629138-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-lpt4s\" (UID: \"96444638-fc69-42cc-ad26-d99bc1629138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:44.839944 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:44.839911 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96444638-fc69-42cc-ad26-d99bc1629138-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-lpt4s\" (UID: \"96444638-fc69-42cc-ad26-d99bc1629138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:44.840131 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:44.840072 2579 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 23 21:52:44.840203 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:44.840134 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96444638-fc69-42cc-ad26-d99bc1629138-tls-certificates podName:96444638-fc69-42cc-ad26-d99bc1629138 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:45.340118459 +0000 UTC m=+78.099189448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/96444638-fc69-42cc-ad26-d99bc1629138-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-lpt4s" (UID: "96444638-fc69-42cc-ad26-d99bc1629138") : secret "prometheus-operator-admission-webhook-tls" not found Jul 23 21:52:45.076628 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:45.076535 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" event={"ID":"6cd66ba2-7eba-46a5-8f66-c25231d3e0be","Type":"ContainerStarted","Data":"abdbc08393745537eadcbc076b96cea15a93b2fd831dca9a077c7868abe14199"} Jul 23 21:52:45.094735 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:45.094686 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-phk9x" podStartSLOduration=33.150592496 podStartE2EDuration="35.094670821s" totalImagesPullingTime="1.944078325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:45.093631185 +0000 UTC m=+77.852702214" watchObservedRunningTime="2026-07-23 21:52:45.094670821 +0000 UTC m=+77.853741831" Jul 23 21:52:45.344276 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:45.344193 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96444638-fc69-42cc-ad26-d99bc1629138-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-lpt4s\" (UID: \"96444638-fc69-42cc-ad26-d99bc1629138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:45.347626 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:45.347595 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96444638-fc69-42cc-ad26-d99bc1629138-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-lpt4s\" (UID: \"96444638-fc69-42cc-ad26-d99bc1629138\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:45.625488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:45.625454 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:45.783391 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:45.783365 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s"] Jul 23 21:52:45.785790 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:45.785764 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96444638_fc69_42cc_ad26_d99bc1629138.slice/crio-7c76b0d217c7fcc1a07c0896f647a8e1040cdf3bc3b7dd952051f66ab29a1a0e WatchSource:0}: Error finding container 7c76b0d217c7fcc1a07c0896f647a8e1040cdf3bc3b7dd952051f66ab29a1a0e: Status 404 returned error can't find the container with id 7c76b0d217c7fcc1a07c0896f647a8e1040cdf3bc3b7dd952051f66ab29a1a0e Jul 23 21:52:46.081467 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:46.081371 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" event={"ID":"96444638-fc69-42cc-ad26-d99bc1629138","Type":"ContainerStarted","Data":"7c76b0d217c7fcc1a07c0896f647a8e1040cdf3bc3b7dd952051f66ab29a1a0e"} Jul 23 21:52:48.090464 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:48.090430 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" event={"ID":"96444638-fc69-42cc-ad26-d99bc1629138","Type":"ContainerStarted","Data":"29e958db0c5a3df2433701f1a9257f76b12554ce5420046c88e791981ea46882"} Jul 23 21:52:48.090977 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:48.090634 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:48.095405 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:48.095383 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" Jul 23 21:52:48.107878 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:48.107834 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-lpt4s" podStartSLOduration=2.610359887 podStartE2EDuration="4.107819283s" totalImagesPullingTime="1.497459396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:48.107279133 +0000 UTC m=+80.866350144" watchObservedRunningTime="2026-07-23 21:52:48.107819283 +0000 UTC m=+80.866890293" Jul 23 21:52:49.054775 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:49.054743 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l4gl4" Jul 23 21:52:50.025896 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.025859 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75b6f8bb76-9ql87"] Jul 23 21:52:50.028626 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.028607 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.031267 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.031248 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 21:52:50.032807 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.032793 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 21:52:50.032923 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.032822 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 21:52:50.032923 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.032796 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 21:52:50.032923 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.032795 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 21:52:50.033067 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.033002 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-srfr5\"" Jul 23 21:52:50.033108 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.033101 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 21:52:50.033151 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.033116 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 21:52:50.043076 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.043043 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b6f8bb76-9ql87"] Jul 23 21:52:50.082003 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.081971 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-oauth-serving-cert\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.082141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.082039 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-oauth-config\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.082141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.082097 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-serving-cert\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.082141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.082115 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-service-ca\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.082141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.082132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftdfd\" (UniqueName: \"kubernetes.io/projected/b099ace2-9a3b-4288-b4c6-de0f3315465a-kube-api-access-ftdfd\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.082274 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.082158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-config\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.183453 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.183403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-oauth-serving-cert\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.183628 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.183499 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-oauth-config\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.183628 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.183528 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-serving-cert\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.183628 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.183550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-service-ca\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.183628 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.183574 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftdfd\" (UniqueName: \"kubernetes.io/projected/b099ace2-9a3b-4288-b4c6-de0f3315465a-kube-api-access-ftdfd\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.183628 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.183625 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-config\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.184279 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.184251 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-service-ca\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.184408 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.184307 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-config\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.184408 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.184307 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-oauth-serving-cert\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.186100 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.186079 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-serving-cert\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.186100 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.186092 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-oauth-config\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.191907 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.191880 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftdfd\" (UniqueName: \"kubernetes.io/projected/b099ace2-9a3b-4288-b4c6-de0f3315465a-kube-api-access-ftdfd\") pod \"console-75b6f8bb76-9ql87\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.284381 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.284305 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:50.286558 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.286527 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5eaafa37-b84a-489c-9d61-75852a35147a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5fhxg\" (UID: \"5eaafa37-b84a-489c-9d61-75852a35147a\") " pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:50.338614 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.338567 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:52:50.466318 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.466287 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75b6f8bb76-9ql87"] Jul 23 21:52:50.469397 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:50.469371 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb099ace2_9a3b_4288_b4c6_de0f3315465a.slice/crio-7dfb7286a22eba52d93787cfc0c87cb3d763eb65a61b024556b1d4d002727aee WatchSource:0}: Error finding container 7dfb7286a22eba52d93787cfc0c87cb3d763eb65a61b024556b1d4d002727aee: Status 404 returned error can't find the container with id 7dfb7286a22eba52d93787cfc0c87cb3d763eb65a61b024556b1d4d002727aee Jul 23 21:52:50.550752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.550662 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jtqd5\"" Jul 23 21:52:50.558891 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.558867 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5fhxg" Jul 23 21:52:50.684640 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:50.684613 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5fhxg"] Jul 23 21:52:50.687092 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:50.687068 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eaafa37_b84a_489c_9d61_75852a35147a.slice/crio-4011aa3a40cf67401f1e6405649887c870be3e1f0da7e11dca205fe9c8608442 WatchSource:0}: Error finding container 4011aa3a40cf67401f1e6405649887c870be3e1f0da7e11dca205fe9c8608442: Status 404 returned error can't find the container with id 4011aa3a40cf67401f1e6405649887c870be3e1f0da7e11dca205fe9c8608442 Jul 23 21:52:51.107975 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:51.107879 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5fhxg" event={"ID":"5eaafa37-b84a-489c-9d61-75852a35147a","Type":"ContainerStarted","Data":"4011aa3a40cf67401f1e6405649887c870be3e1f0da7e11dca205fe9c8608442"} Jul 23 21:52:51.111897 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:51.111836 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b6f8bb76-9ql87" event={"ID":"b099ace2-9a3b-4288-b4c6-de0f3315465a","Type":"ContainerStarted","Data":"7dfb7286a22eba52d93787cfc0c87cb3d763eb65a61b024556b1d4d002727aee"} Jul 23 21:52:53.237316 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.237276 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-blgcc"] Jul 23 21:52:53.240838 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.240814 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p"] Jul 23 21:52:53.240978 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.240932 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.243241 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.243221 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.249321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.249177 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 23 21:52:53.251600 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.251577 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 23 21:52:53.251685 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.251653 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kxbcm\"" Jul 23 21:52:53.251802 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.251737 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 23 21:52:53.251906 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.251833 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mpnnv\"" Jul 23 21:52:53.251974 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.251915 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 23 21:52:53.252012 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.251972 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 23 21:52:53.252105 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.252084 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 21:52:53.258926 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.258906 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-blgcc"] Jul 23 21:52:53.265244 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.265220 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p"] Jul 23 21:52:53.291674 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.291647 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fkrbh"] Jul 23 21:52:53.294684 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.294640 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.298427 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.298393 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 21:52:53.298559 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.298514 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 21:52:53.298623 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.298580 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ktmzs\"" Jul 23 21:52:53.304456 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.304439 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 21:52:53.310176 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310151 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.310298 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310187 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.310298 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310279 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5c237d47-9086-4a90-9d1e-f7f7f025e271-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.310413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1afe055a-60db-4e83-bf38-559fef5044f8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.310413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310335 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c237d47-9086-4a90-9d1e-f7f7f025e271-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.310413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310385 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.310568 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310432 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t6n6\" (UniqueName: \"kubernetes.io/projected/1afe055a-60db-4e83-bf38-559fef5044f8-kube-api-access-5t6n6\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.310568 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310469 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1afe055a-60db-4e83-bf38-559fef5044f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.310568 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310525 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ph7w\" (UniqueName: \"kubernetes.io/projected/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-api-access-9ph7w\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.310568 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.310551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1afe055a-60db-4e83-bf38-559fef5044f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.411431 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411397 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.411431 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411434 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t6n6\" (UniqueName: \"kubernetes.io/projected/1afe055a-60db-4e83-bf38-559fef5044f8-kube-api-access-5t6n6\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411458 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-sys\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411476 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-root\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411495 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1afe055a-60db-4e83-bf38-559fef5044f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411517 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47vld\" (UniqueName: \"kubernetes.io/projected/e9bec962-6b68-4b9a-855f-369e9d47ba09-kube-api-access-47vld\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:53.411542 2579 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411548 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ph7w\" (UniqueName: \"kubernetes.io/projected/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-api-access-9ph7w\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.411665 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:53.411610 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-tls podName:5c237d47-9086-4a90-9d1e-f7f7f025e271 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:53.911589901 +0000 UTC m=+86.670660890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-blgcc" (UID: "5c237d47-9086-4a90-9d1e-f7f7f025e271") : secret "kube-state-metrics-tls" not found Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1afe055a-60db-4e83-bf38-559fef5044f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411744 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-accelerators-collector-config\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411800 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-textfile\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411822 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-wtmp\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411871 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9bec962-6b68-4b9a-855f-369e9d47ba09-metrics-client-ca\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411903 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5c237d47-9086-4a90-9d1e-f7f7f025e271-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411918 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-tls\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411947 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1afe055a-60db-4e83-bf38-559fef5044f8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.412114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.411983 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c237d47-9086-4a90-9d1e-f7f7f025e271-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.412363 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5c237d47-9086-4a90-9d1e-f7f7f025e271-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.412622 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c237d47-9086-4a90-9d1e-f7f7f025e271-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.412681 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.412886 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.412871 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1afe055a-60db-4e83-bf38-559fef5044f8-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.414246 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.414222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1afe055a-60db-4e83-bf38-559fef5044f8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.414413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.414395 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.414636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.414620 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1afe055a-60db-4e83-bf38-559fef5044f8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.424142 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.424116 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t6n6\" (UniqueName: \"kubernetes.io/projected/1afe055a-60db-4e83-bf38-559fef5044f8-kube-api-access-5t6n6\") pod \"openshift-state-metrics-79f6958cdf-hzs7p\" (UID: \"1afe055a-60db-4e83-bf38-559fef5044f8\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.424251 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.424146 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ph7w\" (UniqueName: \"kubernetes.io/projected/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-api-access-9ph7w\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.512775 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47vld\" (UniqueName: \"kubernetes.io/projected/e9bec962-6b68-4b9a-855f-369e9d47ba09-kube-api-access-47vld\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.512775 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-accelerators-collector-config\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512794 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-textfile\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512814 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-wtmp\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9bec962-6b68-4b9a-855f-369e9d47ba09-metrics-client-ca\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-tls\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512936 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.512998 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-sys\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513024 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-root\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513421 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:53.513122 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 21:52:53.513421 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:53.513197 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-tls podName:e9bec962-6b68-4b9a-855f-369e9d47ba09 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:54.013178838 +0000 UTC m=+86.772249843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-tls") pod "node-exporter-fkrbh" (UID: "e9bec962-6b68-4b9a-855f-369e9d47ba09") : secret "node-exporter-tls" not found Jul 23 21:52:53.513421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513212 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-wtmp\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513265 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-root\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-textfile\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513341 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9bec962-6b68-4b9a-855f-369e9d47ba09-sys\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513724 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513441 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-accelerators-collector-config\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.513785 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.513755 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9bec962-6b68-4b9a-855f-369e9d47ba09-metrics-client-ca\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.515598 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.515579 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.522421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.522399 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47vld\" (UniqueName: \"kubernetes.io/projected/e9bec962-6b68-4b9a-855f-369e9d47ba09-kube-api-access-47vld\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:53.560312 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.560282 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" Jul 23 21:52:53.917819 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.917785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:53.920506 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:53.920483 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c237d47-9086-4a90-9d1e-f7f7f025e271-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-blgcc\" (UID: \"5c237d47-9086-4a90-9d1e-f7f7f025e271\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:54.018984 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.018948 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-tls\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:54.021312 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.021287 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e9bec962-6b68-4b9a-855f-369e9d47ba09-node-exporter-tls\") pod \"node-exporter-fkrbh\" (UID: \"e9bec962-6b68-4b9a-855f-369e9d47ba09\") " pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:54.154948 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.154919 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" Jul 23 21:52:54.205502 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.205420 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fkrbh" Jul 23 21:52:54.277722 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.277682 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:52:54.280953 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.280933 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.286154 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286110 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 21:52:54.286274 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286197 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 21:52:54.286274 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286209 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 21:52:54.286377 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286342 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 21:52:54.286477 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286458 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 21:52:54.286641 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286627 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nxdqg\"" Jul 23 21:52:54.286706 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286673 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 21:52:54.286764 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.286712 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 21:52:54.287460 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.287253 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 21:52:54.287460 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.287400 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 21:52:54.299459 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.299436 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:52:54.321645 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321613 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.321808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qwpk\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-kube-api-access-2qwpk\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.321808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.321808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321718 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-web-config\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.321808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321742 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.321808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.321808 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.322177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321816 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.322177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321849 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.322177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321864 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.322177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321881 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.322177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321903 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.322177 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.321938 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-config-out\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.422633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.422593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.422780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.422652 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.422780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.422697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.422780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.422721 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.422780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.422750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423165 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423134 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423283 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423437 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423378 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-config-out\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423437 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423421 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423602 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qwpk\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-kube-api-access-2qwpk\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423602 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423480 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423602 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423502 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-web-config\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423602 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423544 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.423602 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.424173 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.423643 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.424173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:54.423875 2579 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 23 21:52:54.424173 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:54.423941 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls podName:1ad16142-0830-4c62-867c-33a4dd28d486 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:54.923921653 +0000 UTC m=+87.682992645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486") : secret "alertmanager-main-tls" not found Jul 23 21:52:54.424655 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.424627 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.426613 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.426560 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-config-volume\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.427139 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.427095 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.427260 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.427237 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.427502 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.427464 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-web-config\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.428335 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.428309 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.428838 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.428812 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-config-out\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.428929 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.428907 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.429197 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.429173 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.434405 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.434383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qwpk\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-kube-api-access-2qwpk\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.927454 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.927396 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:54.930291 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:54.930262 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:55.196156 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:55.196082 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:52:56.257951 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:56.257676 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9bec962_6b68_4b9a_855f_369e9d47ba09.slice/crio-ec3f2a1dfe77933dec6058c473b5687fe4854f43189cd86b4a2859a449c1618f WatchSource:0}: Error finding container ec3f2a1dfe77933dec6058c473b5687fe4854f43189cd86b4a2859a449c1618f: Status 404 returned error can't find the container with id ec3f2a1dfe77933dec6058c473b5687fe4854f43189cd86b4a2859a449c1618f Jul 23 21:52:56.404997 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:56.404893 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:52:56.411754 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:56.411719 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad16142_0830_4c62_867c_33a4dd28d486.slice/crio-2289ce3725030f6f062cc8f811c604e5e3f8c830283a4bf1c315cec6aee5373f WatchSource:0}: Error finding container 2289ce3725030f6f062cc8f811c604e5e3f8c830283a4bf1c315cec6aee5373f: Status 404 returned error can't find the container with id 2289ce3725030f6f062cc8f811c604e5e3f8c830283a4bf1c315cec6aee5373f Jul 23 21:52:56.411944 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:56.411779 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p"] Jul 23 21:52:56.414732 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:56.414703 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afe055a_60db_4e83_bf38_559fef5044f8.slice/crio-86a5e1afd2d07ac64c2ea048f9decbd03296ead4d4c29504f3cf5dcb61e1c7ad WatchSource:0}: Error finding container 86a5e1afd2d07ac64c2ea048f9decbd03296ead4d4c29504f3cf5dcb61e1c7ad: Status 404 returned error can't find the container with id 86a5e1afd2d07ac64c2ea048f9decbd03296ead4d4c29504f3cf5dcb61e1c7ad Jul 23 21:52:56.440916 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:56.440792 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-blgcc"] Jul 23 21:52:56.445993 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:52:56.445970 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c237d47_9086_4a90_9d1e_f7f7f025e271.slice/crio-09d2b4844427665bc5ffbe524bf175f6fb3ab467742b55e3ee5f18faff613e63 WatchSource:0}: Error finding container 09d2b4844427665bc5ffbe524bf175f6fb3ab467742b55e3ee5f18faff613e63: Status 404 returned error can't find the container with id 09d2b4844427665bc5ffbe524bf175f6fb3ab467742b55e3ee5f18faff613e63 Jul 23 21:52:57.039004 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.038500 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:52:57.148371 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.148333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" event={"ID":"5c237d47-9086-4a90-9d1e-f7f7f025e271","Type":"ContainerStarted","Data":"09d2b4844427665bc5ffbe524bf175f6fb3ab467742b55e3ee5f18faff613e63"} Jul 23 21:52:57.151912 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.151203 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b6f8bb76-9ql87" event={"ID":"b099ace2-9a3b-4288-b4c6-de0f3315465a","Type":"ContainerStarted","Data":"58837771fae56aa323d9347b5f0cf38c6fccefe102469ef3e83426563989c1bb"} Jul 23 21:52:57.154184 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.154158 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fkrbh" event={"ID":"e9bec962-6b68-4b9a-855f-369e9d47ba09","Type":"ContainerStarted","Data":"ec3f2a1dfe77933dec6058c473b5687fe4854f43189cd86b4a2859a449c1618f"} Jul 23 21:52:57.155829 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.155741 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"2289ce3725030f6f062cc8f811c604e5e3f8c830283a4bf1c315cec6aee5373f"} Jul 23 21:52:57.166904 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.161705 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" event={"ID":"1afe055a-60db-4e83-bf38-559fef5044f8","Type":"ContainerStarted","Data":"3bdf60f7bd886c87166d13fb8b3b6ebe65b9258a958045629bc461fea1d90321"} Jul 23 21:52:57.166904 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.161804 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" event={"ID":"1afe055a-60db-4e83-bf38-559fef5044f8","Type":"ContainerStarted","Data":"c8384e115a2e095c2f72a2938c9b4ea7d83b6ac6e29000ffde4f598f1d683d0d"} Jul 23 21:52:57.166904 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.161840 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" event={"ID":"1afe055a-60db-4e83-bf38-559fef5044f8","Type":"ContainerStarted","Data":"86a5e1afd2d07ac64c2ea048f9decbd03296ead4d4c29504f3cf5dcb61e1c7ad"} Jul 23 21:52:57.166904 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.164634 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5fhxg" event={"ID":"5eaafa37-b84a-489c-9d61-75852a35147a","Type":"ContainerStarted","Data":"5b290ed5822ad29714b27aeb1f269593862456139417383bb46461c693890b52"} Jul 23 21:52:57.180530 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.179447 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75b6f8bb76-9ql87" podStartSLOduration=1.444558905 podStartE2EDuration="7.179429941s" totalImagesPullingTime="5.734871036s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:57.179079895 +0000 UTC m=+89.938150909" watchObservedRunningTime="2026-07-23 21:52:57.179429941 +0000 UTC m=+89.938500956" Jul 23 21:52:57.988210 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.988171 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75b6f8bb76-9ql87"] Jul 23 21:52:57.989907 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.989877 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w"] Jul 23 21:52:57.993766 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.993744 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:57.996530 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.996511 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-r6tns\"" Jul 23 21:52:57.996982 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:57.996956 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 21:52:58.003247 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.003136 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w"] Jul 23 21:52:58.025436 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.024927 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b9567c8f5-g9xhh"] Jul 23 21:52:58.028942 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.028918 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.038620 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.038585 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b9567c8f5-g9xhh"] Jul 23 21:52:58.041173 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.041141 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 21:52:58.054484 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.054374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-config\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.054484 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.054450 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-trusted-ca-bundle\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.055041 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.054822 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqvl2\" (UniqueName: \"kubernetes.io/projected/7ad871d9-5dd1-4658-8bb1-9bba843196a5-kube-api-access-xqvl2\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.055276 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.055248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-oauth-serving-cert\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.055398 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.055377 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-service-ca\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.055482 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.055465 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-oauth-config\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.055538 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.055515 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8069444e-71d4-4297-9fc1-4d12103d1dee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.055589 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.055545 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vjr\" (UniqueName: \"kubernetes.io/projected/8069444e-71d4-4297-9fc1-4d12103d1dee-kube-api-access-z5vjr\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.055638 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.055584 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-serving-cert\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156217 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-oauth-config\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156493 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156231 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8069444e-71d4-4297-9fc1-4d12103d1dee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.156493 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156261 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vjr\" (UniqueName: \"kubernetes.io/projected/8069444e-71d4-4297-9fc1-4d12103d1dee-kube-api-access-z5vjr\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.156493 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156287 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-serving-cert\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156493 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:58.156375 2579 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 23 21:52:58.156493 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:52:58.156442 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8069444e-71d4-4297-9fc1-4d12103d1dee-monitoring-plugin-cert podName:8069444e-71d4-4297-9fc1-4d12103d1dee nodeName:}" failed. No retries permitted until 2026-07-23 21:52:58.656422752 +0000 UTC m=+91.415493754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8069444e-71d4-4297-9fc1-4d12103d1dee-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-scn6w" (UID: "8069444e-71d4-4297-9fc1-4d12103d1dee") : secret "monitoring-plugin-cert" not found Jul 23 21:52:58.156493 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156465 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-config\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156834 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-trusted-ca-bundle\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156834 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156534 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqvl2\" (UniqueName: \"kubernetes.io/projected/7ad871d9-5dd1-4658-8bb1-9bba843196a5-kube-api-access-xqvl2\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156834 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156590 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-oauth-serving-cert\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.156834 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.156643 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-service-ca\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.157372 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.157346 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-config\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.157594 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.157575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-oauth-serving-cert\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.157952 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.157923 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-service-ca\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.158341 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.158296 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-trusted-ca-bundle\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.159459 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.159415 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-oauth-config\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.159459 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.159425 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-serving-cert\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.167251 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.167205 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vjr\" (UniqueName: \"kubernetes.io/projected/8069444e-71d4-4297-9fc1-4d12103d1dee-kube-api-access-z5vjr\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.167988 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.167946 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqvl2\" (UniqueName: \"kubernetes.io/projected/7ad871d9-5dd1-4658-8bb1-9bba843196a5-kube-api-access-xqvl2\") pod \"console-6b9567c8f5-g9xhh\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.342848 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.342765 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:52:58.662989 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.662947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8069444e-71d4-4297-9fc1-4d12103d1dee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.665705 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.665677 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8069444e-71d4-4297-9fc1-4d12103d1dee-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-scn6w\" (UID: \"8069444e-71d4-4297-9fc1-4d12103d1dee\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:52:58.906416 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:52:58.906385 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:53:00.339394 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:00.339350 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:53:00.477652 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:00.477627 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b9567c8f5-g9xhh"] Jul 23 21:53:00.489733 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:53:00.489690 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad871d9_5dd1_4658_8bb1_9bba843196a5.slice/crio-c709ed46b60ef60f2c45a274f3668e0660281fb01b4feea95e7c7572528d54e4 WatchSource:0}: Error finding container c709ed46b60ef60f2c45a274f3668e0660281fb01b4feea95e7c7572528d54e4: Status 404 returned error can't find the container with id c709ed46b60ef60f2c45a274f3668e0660281fb01b4feea95e7c7572528d54e4 Jul 23 21:53:00.598779 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:00.598515 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w"] Jul 23 21:53:00.623177 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:53:00.623144 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8069444e_71d4_4297_9fc1_4d12103d1dee.slice/crio-9083f71906ca0e56403ee126f6eab8a54b1a34189a60a861aa3d5698adc45c9a WatchSource:0}: Error finding container 9083f71906ca0e56403ee126f6eab8a54b1a34189a60a861aa3d5698adc45c9a: Status 404 returned error can't find the container with id 9083f71906ca0e56403ee126f6eab8a54b1a34189a60a861aa3d5698adc45c9a Jul 23 21:53:01.195380 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.195012 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" event={"ID":"5c237d47-9086-4a90-9d1e-f7f7f025e271","Type":"ContainerStarted","Data":"2ffed13339e675f1275ad3284d2b89141f2c1f0706541ca947491858c5e7d4c3"} Jul 23 21:53:01.195380 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.195102 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" event={"ID":"5c237d47-9086-4a90-9d1e-f7f7f025e271","Type":"ContainerStarted","Data":"aa947b66ea6d6f8d9ebfcf5c0b31b19527db2e8021a8d152958caa1c601b4a07"} Jul 23 21:53:01.195380 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.195135 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" event={"ID":"5c237d47-9086-4a90-9d1e-f7f7f025e271","Type":"ContainerStarted","Data":"0043288e2549d6e8335d6624398ac60d37b474e0614d053b7d6c86c65366db39"} Jul 23 21:53:01.196933 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.196901 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" event={"ID":"8069444e-71d4-4297-9fc1-4d12103d1dee","Type":"ContainerStarted","Data":"9083f71906ca0e56403ee126f6eab8a54b1a34189a60a861aa3d5698adc45c9a"} Jul 23 21:53:01.200546 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.200520 2579 generic.go:356] "Generic (PLEG): container finished" podID="e9bec962-6b68-4b9a-855f-369e9d47ba09" containerID="2b4edaaea52c1fcb3342a6f8a3f865e8bb7c3ac71de48fa23e5ca0cd16976de3" exitCode=0 Jul 23 21:53:01.200652 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.200598 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fkrbh" event={"ID":"e9bec962-6b68-4b9a-855f-369e9d47ba09","Type":"ContainerDied","Data":"2b4edaaea52c1fcb3342a6f8a3f865e8bb7c3ac71de48fa23e5ca0cd16976de3"} Jul 23 21:53:01.202459 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.202416 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25" exitCode=0 Jul 23 21:53:01.202556 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.202509 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25"} Jul 23 21:53:01.205085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.205043 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9567c8f5-g9xhh" event={"ID":"7ad871d9-5dd1-4658-8bb1-9bba843196a5","Type":"ContainerStarted","Data":"104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45"} Jul 23 21:53:01.205184 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.205105 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9567c8f5-g9xhh" event={"ID":"7ad871d9-5dd1-4658-8bb1-9bba843196a5","Type":"ContainerStarted","Data":"c709ed46b60ef60f2c45a274f3668e0660281fb01b4feea95e7c7572528d54e4"} Jul 23 21:53:01.208348 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.208322 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" event={"ID":"1afe055a-60db-4e83-bf38-559fef5044f8","Type":"ContainerStarted","Data":"2e034ae518bff27ca8ba0cb76bc8b21910d7d8df84f1cddc737e3fdb0c203e4a"} Jul 23 21:53:01.211401 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.211378 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5fhxg" event={"ID":"5eaafa37-b84a-489c-9d61-75852a35147a","Type":"ContainerStarted","Data":"1634376a9c9c20fa79a531b908061ecfac3de5fd3487e75358a4edeae9c7d46e"} Jul 23 21:53:01.258512 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.258439 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-blgcc" podStartSLOduration=4.275597366 podStartE2EDuration="8.258414567s" totalImagesPullingTime="3.982817201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:01.220714683 +0000 UTC m=+93.979785694" watchObservedRunningTime="2026-07-23 21:53:01.258414567 +0000 UTC m=+94.017485579" Jul 23 21:53:01.286697 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:01.286636 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b9567c8f5-g9xhh" podStartSLOduration=4.286615684 podStartE2EDuration="4.286615684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:01.284350048 +0000 UTC m=+94.043421062" watchObservedRunningTime="2026-07-23 21:53:01.286615684 +0000 UTC m=+94.045686695" Jul 23 21:53:02.221921 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.221868 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fkrbh" event={"ID":"e9bec962-6b68-4b9a-855f-369e9d47ba09","Type":"ContainerStarted","Data":"774e92d632a4c8be3555612b98d15b33e38b34cb7c0a39f373b0ee97a1319c28"} Jul 23 21:53:02.222394 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.221940 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fkrbh" event={"ID":"e9bec962-6b68-4b9a-855f-369e9d47ba09","Type":"ContainerStarted","Data":"3b560743ff7c2f8d917f77dc52217050f7966004a7682d32a14d33f9140413ef"} Jul 23 21:53:02.223902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.223806 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" event={"ID":"8069444e-71d4-4297-9fc1-4d12103d1dee","Type":"ContainerStarted","Data":"eb98d8583fcbf8fb7f1f03cffa655551afd1a8f4c083c67136b6a0a352774494"} Jul 23 21:53:02.249085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.248993 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fkrbh" podStartSLOduration=5.686292878 podStartE2EDuration="9.24896098s" totalImagesPullingTime="3.459328606s" totalInitContainerRuntime="103.339496ms" isStatefulPod=true podCreationTimestamp="2026-07-23 21:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:02.246028035 +0000 UTC m=+95.005099071" watchObservedRunningTime="2026-07-23 21:53:02.24896098 +0000 UTC m=+95.008031992" Jul 23 21:53:02.249444 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.249404 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-hzs7p" podStartSLOduration=5.369432655 podStartE2EDuration="9.249393379s" totalImagesPullingTime="3.879960724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:01.336417292 +0000 UTC m=+94.095488305" watchObservedRunningTime="2026-07-23 21:53:02.249393379 +0000 UTC m=+95.008464391" Jul 23 21:53:02.265583 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.265523 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" podStartSLOduration=4.048972923 podStartE2EDuration="5.265506006s" totalImagesPullingTime="1.216533083s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:02.264107012 +0000 UTC m=+95.023178025" watchObservedRunningTime="2026-07-23 21:53:02.265506006 +0000 UTC m=+95.024577018" Jul 23 21:53:02.590979 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:02.590909 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-765678454c-xhg9v"] Jul 23 21:53:03.229190 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:03.229146 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:53:03.235235 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:03.235211 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-scn6w" Jul 23 21:53:05.020073 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.020031 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kncqt" Jul 23 21:53:05.240102 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.240037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98"} Jul 23 21:53:05.240292 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.240144 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5"} Jul 23 21:53:05.240292 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.240178 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f"} Jul 23 21:53:05.240292 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.240203 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4"} Jul 23 21:53:05.240292 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.240235 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb"} Jul 23 21:53:05.242175 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.242151 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5fhxg" event={"ID":"5eaafa37-b84a-489c-9d61-75852a35147a","Type":"ContainerStarted","Data":"4d24f45bdf64a53618e1ebff7ef980ecc2ec7b620323aedf011e359b694cac02"} Jul 23 21:53:05.264454 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:05.264410 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5fhxg" podStartSLOduration=33.532724576 podStartE2EDuration="47.264390791s" totalImagesPullingTime="13.731666215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:52:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:05.262496955 +0000 UTC m=+98.021567999" watchObservedRunningTime="2026-07-23 21:53:05.264390791 +0000 UTC m=+98.023461801" Jul 23 21:53:06.250020 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:06.249983 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerStarted","Data":"18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821"} Jul 23 21:53:06.282750 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:06.282703 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.330396926 podStartE2EDuration="12.282686303s" totalImagesPullingTime="7.922883483s" totalInitContainerRuntime="29.405894ms" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:06.281639701 +0000 UTC m=+99.040710711" watchObservedRunningTime="2026-07-23 21:53:06.282686303 +0000 UTC m=+99.041757314" Jul 23 21:53:08.342954 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:08.342916 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:53:08.343370 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:08.342994 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:53:08.347852 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:08.347829 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:53:09.269423 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:09.269395 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:53:24.205215 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.205177 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75b6f8bb76-9ql87" podUID="b099ace2-9a3b-4288-b4c6-de0f3315465a" containerName="console" containerID="cri-o://58837771fae56aa323d9347b5f0cf38c6fccefe102469ef3e83426563989c1bb" gracePeriod=15 Jul 23 21:53:24.332949 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.332927 2579 generic.go:356] "Generic (PLEG): container finished" podID="b099ace2-9a3b-4288-b4c6-de0f3315465a" containerID="58837771fae56aa323d9347b5f0cf38c6fccefe102469ef3e83426563989c1bb" exitCode=0 Jul 23 21:53:24.333014 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.332995 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b6f8bb76-9ql87" event={"ID":"b099ace2-9a3b-4288-b4c6-de0f3315465a","Type":"ContainerDied","Data":"58837771fae56aa323d9347b5f0cf38c6fccefe102469ef3e83426563989c1bb"} Jul 23 21:53:24.487421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.487354 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:53:24.599240 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599205 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-service-ca\") pod \"b099ace2-9a3b-4288-b4c6-de0f3315465a\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " Jul 23 21:53:24.599240 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599242 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-config\") pod \"b099ace2-9a3b-4288-b4c6-de0f3315465a\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " Jul 23 21:53:24.599461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599272 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-oauth-config\") pod \"b099ace2-9a3b-4288-b4c6-de0f3315465a\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " Jul 23 21:53:24.599461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599339 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-oauth-serving-cert\") pod \"b099ace2-9a3b-4288-b4c6-de0f3315465a\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " Jul 23 21:53:24.599461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599358 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftdfd\" (UniqueName: \"kubernetes.io/projected/b099ace2-9a3b-4288-b4c6-de0f3315465a-kube-api-access-ftdfd\") pod \"b099ace2-9a3b-4288-b4c6-de0f3315465a\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " Jul 23 21:53:24.599461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599393 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-serving-cert\") pod \"b099ace2-9a3b-4288-b4c6-de0f3315465a\" (UID: \"b099ace2-9a3b-4288-b4c6-de0f3315465a\") " Jul 23 21:53:24.599699 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599670 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-config" (OuterVolumeSpecName: "console-config") pod "b099ace2-9a3b-4288-b4c6-de0f3315465a" (UID: "b099ace2-9a3b-4288-b4c6-de0f3315465a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:24.599774 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599678 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-service-ca" (OuterVolumeSpecName: "service-ca") pod "b099ace2-9a3b-4288-b4c6-de0f3315465a" (UID: "b099ace2-9a3b-4288-b4c6-de0f3315465a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:24.599774 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.599728 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b099ace2-9a3b-4288-b4c6-de0f3315465a" (UID: "b099ace2-9a3b-4288-b4c6-de0f3315465a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:24.601671 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.601634 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b099ace2-9a3b-4288-b4c6-de0f3315465a" (UID: "b099ace2-9a3b-4288-b4c6-de0f3315465a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:53:24.601671 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.601654 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b099ace2-9a3b-4288-b4c6-de0f3315465a" (UID: "b099ace2-9a3b-4288-b4c6-de0f3315465a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:53:24.601812 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.601634 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b099ace2-9a3b-4288-b4c6-de0f3315465a-kube-api-access-ftdfd" (OuterVolumeSpecName: "kube-api-access-ftdfd") pod "b099ace2-9a3b-4288-b4c6-de0f3315465a" (UID: "b099ace2-9a3b-4288-b4c6-de0f3315465a"). InnerVolumeSpecName "kube-api-access-ftdfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:24.700935 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.700902 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:24.700935 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.700928 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-service-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:24.700935 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.700937 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:24.700935 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.700945 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b099ace2-9a3b-4288-b4c6-de0f3315465a-console-oauth-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:24.701206 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.700954 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b099ace2-9a3b-4288-b4c6-de0f3315465a-oauth-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:24.701206 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:24.700962 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftdfd\" (UniqueName: \"kubernetes.io/projected/b099ace2-9a3b-4288-b4c6-de0f3315465a-kube-api-access-ftdfd\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:25.338605 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:25.338574 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75b6f8bb76-9ql87" event={"ID":"b099ace2-9a3b-4288-b4c6-de0f3315465a","Type":"ContainerDied","Data":"7dfb7286a22eba52d93787cfc0c87cb3d763eb65a61b024556b1d4d002727aee"} Jul 23 21:53:25.338605 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:25.338591 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75b6f8bb76-9ql87" Jul 23 21:53:25.339068 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:25.338632 2579 scope.go:117] "RemoveContainer" containerID="58837771fae56aa323d9347b5f0cf38c6fccefe102469ef3e83426563989c1bb" Jul 23 21:53:25.360421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:25.360395 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75b6f8bb76-9ql87"] Jul 23 21:53:25.364490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:25.364470 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75b6f8bb76-9ql87"] Jul 23 21:53:25.678312 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:25.678276 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b099ace2-9a3b-4288-b4c6-de0f3315465a" path="/var/lib/kubelet/pods/b099ace2-9a3b-4288-b4c6-de0f3315465a/volumes" Jul 23 21:53:27.620200 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:27.620168 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-765678454c-xhg9v" podUID="84b492fd-77e2-4ce2-9a18-5391c94b98d5" containerName="registry" containerID="cri-o://0e768e36483b0913512c3901c71e3748ab3c3b213bf7f2593fda8dd33071aa07" gracePeriod=30 Jul 23 21:53:28.353348 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.353316 2579 generic.go:356] "Generic (PLEG): container finished" podID="8c0c96d0-bbea-4969-a466-5c46ab6285f8" containerID="ab0d60d0b5e70123704d0646210d913492e105c1ad2a1b282086dc111ef8765a" exitCode=0 Jul 23 21:53:28.353522 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.353390 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" event={"ID":"8c0c96d0-bbea-4969-a466-5c46ab6285f8","Type":"ContainerDied","Data":"ab0d60d0b5e70123704d0646210d913492e105c1ad2a1b282086dc111ef8765a"} Jul 23 21:53:28.353753 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.353732 2579 scope.go:117] "RemoveContainer" containerID="ab0d60d0b5e70123704d0646210d913492e105c1ad2a1b282086dc111ef8765a" Jul 23 21:53:28.354942 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.354919 2579 generic.go:356] "Generic (PLEG): container finished" podID="00df7f87-13d8-4a59-abda-a4f46db66d9a" containerID="6fd8f0f4bc944ddea8e67cba9fb7328cc78770caa069e3829888a377fe105f93" exitCode=0 Jul 23 21:53:28.355043 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.354991 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" event={"ID":"00df7f87-13d8-4a59-abda-a4f46db66d9a","Type":"ContainerDied","Data":"6fd8f0f4bc944ddea8e67cba9fb7328cc78770caa069e3829888a377fe105f93"} Jul 23 21:53:28.355284 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.355265 2579 scope.go:117] "RemoveContainer" containerID="6fd8f0f4bc944ddea8e67cba9fb7328cc78770caa069e3829888a377fe105f93" Jul 23 21:53:28.356583 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.356550 2579 generic.go:356] "Generic (PLEG): container finished" podID="84b492fd-77e2-4ce2-9a18-5391c94b98d5" containerID="0e768e36483b0913512c3901c71e3748ab3c3b213bf7f2593fda8dd33071aa07" exitCode=0 Jul 23 21:53:28.356661 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.356627 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765678454c-xhg9v" event={"ID":"84b492fd-77e2-4ce2-9a18-5391c94b98d5","Type":"ContainerDied","Data":"0e768e36483b0913512c3901c71e3748ab3c3b213bf7f2593fda8dd33071aa07"} Jul 23 21:53:28.412818 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.412798 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:53:28.537101 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.536996 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-trusted-ca\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537259 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537152 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-certificates\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537259 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537191 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-image-registry-private-configuration\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537259 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537244 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84b492fd-77e2-4ce2-9a18-5391c94b98d5-ca-trust-extracted\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537412 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537283 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-bound-sa-token\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537412 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537312 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537412 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537335 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtcbx\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-kube-api-access-gtcbx\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537412 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537387 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-installation-pull-secrets\") pod \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\" (UID: \"84b492fd-77e2-4ce2-9a18-5391c94b98d5\") " Jul 23 21:53:28.537596 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537450 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:28.538347 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537794 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-trusted-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.538347 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.537459 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:28.540584 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.540527 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:53:28.540584 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.540557 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:28.540733 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.540614 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-kube-api-access-gtcbx" (OuterVolumeSpecName: "kube-api-access-gtcbx") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "kube-api-access-gtcbx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:28.541185 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.541166 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:28.541868 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.541850 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:53:28.548572 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.548546 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84b492fd-77e2-4ce2-9a18-5391c94b98d5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "84b492fd-77e2-4ce2-9a18-5391c94b98d5" (UID: "84b492fd-77e2-4ce2-9a18-5391c94b98d5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:53:28.638205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638175 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-certificates\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.638205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638198 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-image-registry-private-configuration\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.638205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638208 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/84b492fd-77e2-4ce2-9a18-5391c94b98d5-ca-trust-extracted\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.638731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638217 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-bound-sa-token\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.638731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638225 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-registry-tls\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.638731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638233 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtcbx\" (UniqueName: \"kubernetes.io/projected/84b492fd-77e2-4ce2-9a18-5391c94b98d5-kube-api-access-gtcbx\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:28.638731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:28.638241 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/84b492fd-77e2-4ce2-9a18-5391c94b98d5-installation-pull-secrets\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:53:29.362340 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.362304 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-6h9vg" event={"ID":"8c0c96d0-bbea-4969-a466-5c46ab6285f8","Type":"ContainerStarted","Data":"34af555dd35c4da2c733cc1c63e436a164743a3dcc3c64f8f4060bfb0fd47e01"} Jul 23 21:53:29.364067 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.364037 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-8xv62" event={"ID":"00df7f87-13d8-4a59-abda-a4f46db66d9a","Type":"ContainerStarted","Data":"7a87ce0508a0c3b7d2248965589184588fb6dd1e27db77ad99060e7819baa2c6"} Jul 23 21:53:29.365257 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.365234 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-765678454c-xhg9v" event={"ID":"84b492fd-77e2-4ce2-9a18-5391c94b98d5","Type":"ContainerDied","Data":"34cffc13e01641613c80e6e98123098ccdea043cd8884a5d771536ff9302fb50"} Jul 23 21:53:29.365257 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.365247 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-765678454c-xhg9v" Jul 23 21:53:29.365450 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.365272 2579 scope.go:117] "RemoveContainer" containerID="0e768e36483b0913512c3901c71e3748ab3c3b213bf7f2593fda8dd33071aa07" Jul 23 21:53:29.400402 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.400368 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-765678454c-xhg9v"] Jul 23 21:53:29.406127 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.406102 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-765678454c-xhg9v"] Jul 23 21:53:29.678014 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:29.677939 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84b492fd-77e2-4ce2-9a18-5391c94b98d5" path="/var/lib/kubelet/pods/84b492fd-77e2-4ce2-9a18-5391c94b98d5/volumes" Jul 23 21:53:33.383281 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:33.383245 2579 generic.go:356] "Generic (PLEG): container finished" podID="59ed4961-d552-4186-8e13-31f800553994" containerID="51bc84e656059c36683ed98ff53f623045fcb7c1309a98ecb1104e9b902434d5" exitCode=0 Jul 23 21:53:33.383703 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:33.383316 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-42jjx" event={"ID":"59ed4961-d552-4186-8e13-31f800553994","Type":"ContainerDied","Data":"51bc84e656059c36683ed98ff53f623045fcb7c1309a98ecb1104e9b902434d5"} Jul 23 21:53:33.383703 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:33.383637 2579 scope.go:117] "RemoveContainer" containerID="51bc84e656059c36683ed98ff53f623045fcb7c1309a98ecb1104e9b902434d5" Jul 23 21:53:34.390641 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:53:34.390605 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-42jjx" event={"ID":"59ed4961-d552-4186-8e13-31f800553994","Type":"ContainerStarted","Data":"1b4ea93d0625ec6a9412ab6e93798777e2a8c5af6bfcaf4e070afb97323196a3"} Jul 23 21:54:13.546278 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546240 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:54:13.546826 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546524 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="alertmanager" containerID="cri-o://42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb" gracePeriod=120 Jul 23 21:54:13.546826 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546556 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy" containerID="cri-o://3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5" gracePeriod=120 Jul 23 21:54:13.546826 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546582 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="prom-label-proxy" containerID="cri-o://18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821" gracePeriod=120 Jul 23 21:54:13.546826 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546590 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="config-reloader" containerID="cri-o://1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4" gracePeriod=120 Jul 23 21:54:13.546826 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546622 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-metric" containerID="cri-o://eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98" gracePeriod=120 Jul 23 21:54:13.546826 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:13.546703 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-web" containerID="cri-o://798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f" gracePeriod=120 Jul 23 21:54:14.559103 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559067 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821" exitCode=0 Jul 23 21:54:14.559103 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559082 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821"} Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559152 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98"} Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559092 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98" exitCode=0 Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559185 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5" exitCode=0 Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559197 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4" exitCode=0 Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559204 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb" exitCode=0 Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559260 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5"} Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559317 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4"} Jul 23 21:54:14.559534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.559343 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb"} Jul 23 21:54:14.786797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.786773 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:14.855165 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855141 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-main-db\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855337 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855176 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-config-out\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855337 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855195 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-metrics-client-ca\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855337 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855228 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-tls-assets\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855337 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855249 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-web\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855337 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855308 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-web-config\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855337 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855332 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-config-volume\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855358 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855398 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qwpk\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-kube-api-access-2qwpk\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855425 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855460 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-cluster-tls-config\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855490 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-trusted-ca-bundle\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855523 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy\") pod \"1ad16142-0830-4c62-867c-33a4dd28d486\" (UID: \"1ad16142-0830-4c62-867c-33a4dd28d486\") " Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855527 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:54:14.855647 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855559 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:54:14.856048 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855792 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-main-db\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.856048 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.855814 2579 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-metrics-client-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.856794 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.856738 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:54:14.858492 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.858451 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:54:14.858781 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.858670 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-kube-api-access-2qwpk" (OuterVolumeSpecName: "kube-api-access-2qwpk") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "kube-api-access-2qwpk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:54:14.858781 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.858743 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-config-out" (OuterVolumeSpecName: "config-out") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:54:14.859723 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.859664 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-config-volume" (OuterVolumeSpecName: "config-volume") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.859854 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.859813 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.859854 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.859831 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.860176 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.860148 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.860685 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.860655 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.863630 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.863513 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.871323 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.871268 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-web-config" (OuterVolumeSpecName: "web-config") pod "1ad16142-0830-4c62-867c-33a4dd28d486" (UID: "1ad16142-0830-4c62-867c-33a4dd28d486"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:14.956437 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956407 2579 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-web-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956437 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956432 2579 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-config-volume\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956437 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956441 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-main-tls\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956450 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qwpk\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-kube-api-access-2qwpk\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956460 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956468 2579 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-cluster-tls-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956477 2579 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ad16142-0830-4c62-867c-33a4dd28d486-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956485 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956493 2579 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1ad16142-0830-4c62-867c-33a4dd28d486-config-out\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956503 2579 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1ad16142-0830-4c62-867c-33a4dd28d486-tls-assets\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:14.956634 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:14.956510 2579 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1ad16142-0830-4c62-867c-33a4dd28d486-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:15.571003 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.570972 2579 generic.go:356] "Generic (PLEG): container finished" podID="1ad16142-0830-4c62-867c-33a4dd28d486" containerID="798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f" exitCode=0 Jul 23 21:54:15.571482 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.571068 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.571482 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.571074 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f"} Jul 23 21:54:15.571482 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.571194 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1ad16142-0830-4c62-867c-33a4dd28d486","Type":"ContainerDied","Data":"2289ce3725030f6f062cc8f811c604e5e3f8c830283a4bf1c315cec6aee5373f"} Jul 23 21:54:15.571482 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.571244 2579 scope.go:117] "RemoveContainer" containerID="18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821" Jul 23 21:54:15.579414 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.579354 2579 scope.go:117] "RemoveContainer" containerID="eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98" Jul 23 21:54:15.586472 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.586457 2579 scope.go:117] "RemoveContainer" containerID="3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5" Jul 23 21:54:15.593198 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.593171 2579 scope.go:117] "RemoveContainer" containerID="798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f" Jul 23 21:54:15.598104 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.598081 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:54:15.600029 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.600012 2579 scope.go:117] "RemoveContainer" containerID="1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4" Jul 23 21:54:15.604889 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.604862 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:54:15.607392 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.607378 2579 scope.go:117] "RemoveContainer" containerID="42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb" Jul 23 21:54:15.614103 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.614085 2579 scope.go:117] "RemoveContainer" containerID="0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25" Jul 23 21:54:15.621161 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621147 2579 scope.go:117] "RemoveContainer" containerID="18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821" Jul 23 21:54:15.621392 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.621374 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821\": container with ID starting with 18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821 not found: ID does not exist" containerID="18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821" Jul 23 21:54:15.621442 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621402 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821"} err="failed to get container status \"18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821\": rpc error: code = NotFound desc = could not find container \"18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821\": container with ID starting with 18cb0dca9ed1a101e6a986b8e34167dba010c0a7578bf2f24feb9cfd44cf1821 not found: ID does not exist" Jul 23 21:54:15.621442 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621433 2579 scope.go:117] "RemoveContainer" containerID="eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98" Jul 23 21:54:15.621674 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.621657 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98\": container with ID starting with eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98 not found: ID does not exist" containerID="eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98" Jul 23 21:54:15.621737 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621685 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98"} err="failed to get container status \"eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98\": rpc error: code = NotFound desc = could not find container \"eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98\": container with ID starting with eec272e68a5f18dc00c94c9da214125ece929e2010e155e3756983dff7c50c98 not found: ID does not exist" Jul 23 21:54:15.621737 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621709 2579 scope.go:117] "RemoveContainer" containerID="3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5" Jul 23 21:54:15.621924 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.621908 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5\": container with ID starting with 3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5 not found: ID does not exist" containerID="3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5" Jul 23 21:54:15.621976 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621928 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5"} err="failed to get container status \"3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5\": rpc error: code = NotFound desc = could not find container \"3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5\": container with ID starting with 3ddb776e64bc991bd463840af3a7df1ade0c14b1a5e5f8eee19102df4553d8f5 not found: ID does not exist" Jul 23 21:54:15.621976 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.621941 2579 scope.go:117] "RemoveContainer" containerID="798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f" Jul 23 21:54:15.622164 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.622145 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f\": container with ID starting with 798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f not found: ID does not exist" containerID="798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f" Jul 23 21:54:15.622270 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622165 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f"} err="failed to get container status \"798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f\": rpc error: code = NotFound desc = could not find container \"798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f\": container with ID starting with 798bdef69611bf5d94ecfa6bd3dbf66e8fe289d5efadb6cb3794a030da6c6f2f not found: ID does not exist" Jul 23 21:54:15.622270 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622177 2579 scope.go:117] "RemoveContainer" containerID="1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4" Jul 23 21:54:15.622401 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.622387 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4\": container with ID starting with 1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4 not found: ID does not exist" containerID="1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4" Jul 23 21:54:15.622461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622403 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4"} err="failed to get container status \"1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4\": rpc error: code = NotFound desc = could not find container \"1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4\": container with ID starting with 1f1f411876af1a43a20693de2905f8a9c20eeab3b2383d78d34c39387fb549a4 not found: ID does not exist" Jul 23 21:54:15.622461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622414 2579 scope.go:117] "RemoveContainer" containerID="42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb" Jul 23 21:54:15.622621 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.622608 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb\": container with ID starting with 42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb not found: ID does not exist" containerID="42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb" Jul 23 21:54:15.622676 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622622 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb"} err="failed to get container status \"42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb\": rpc error: code = NotFound desc = could not find container \"42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb\": container with ID starting with 42fd752135d0fe12f5e0ca3c64480a01a73dcde57a291d4a216cf90f4fccd4bb not found: ID does not exist" Jul 23 21:54:15.622676 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622632 2579 scope.go:117] "RemoveContainer" containerID="0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25" Jul 23 21:54:15.622814 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:15.622800 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25\": container with ID starting with 0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25 not found: ID does not exist" containerID="0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25" Jul 23 21:54:15.622864 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.622815 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25"} err="failed to get container status \"0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25\": rpc error: code = NotFound desc = could not find container \"0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25\": container with ID starting with 0633d2d7078a8f4938be1f474b5ef5f12d0e17aa18ff2f08052099d3effbbb25 not found: ID does not exist" Jul 23 21:54:15.645116 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645093 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:54:15.645421 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645410 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84b492fd-77e2-4ce2-9a18-5391c94b98d5" containerName="registry" Jul 23 21:54:15.645462 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645422 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="84b492fd-77e2-4ce2-9a18-5391c94b98d5" containerName="registry" Jul 23 21:54:15.645462 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645431 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-web" Jul 23 21:54:15.645462 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645437 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-web" Jul 23 21:54:15.645462 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645443 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="prom-label-proxy" Jul 23 21:54:15.645462 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645449 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="prom-label-proxy" Jul 23 21:54:15.645462 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645459 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-metric" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645464 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-metric" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645473 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645478 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645488 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="config-reloader" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645493 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="config-reloader" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645500 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="alertmanager" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645505 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="alertmanager" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645511 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="init-config-reloader" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645516 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="init-config-reloader" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645521 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b099ace2-9a3b-4288-b4c6-de0f3315465a" containerName="console" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645526 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b099ace2-9a3b-4288-b4c6-de0f3315465a" containerName="console" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645580 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="alertmanager" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645586 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="config-reloader" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645591 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-web" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645596 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="prom-label-proxy" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645602 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy-metric" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645608 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b099ace2-9a3b-4288-b4c6-de0f3315465a" containerName="console" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645613 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" containerName="kube-rbac-proxy" Jul 23 21:54:15.645636 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.645621 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="84b492fd-77e2-4ce2-9a18-5391c94b98d5" containerName="registry" Jul 23 21:54:15.650692 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.650678 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.653494 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.653476 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nxdqg\"" Jul 23 21:54:15.653784 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.653767 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 21:54:15.653891 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.653786 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 21:54:15.653891 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.653807 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 21:54:15.653891 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.653829 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 21:54:15.654026 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.653897 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 21:54:15.654203 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.654188 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 21:54:15.654284 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.654229 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 21:54:15.654353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.654326 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 21:54:15.659555 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.659538 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 21:54:15.661563 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661542 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661642 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661573 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661642 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661591 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-config-volume\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661642 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661608 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13a90b1a-e89d-4c50-87fe-5b08a0018d79-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661709 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gbrj\" (UniqueName: \"kubernetes.io/projected/13a90b1a-e89d-4c50-87fe-5b08a0018d79-kube-api-access-5gbrj\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661752 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-web-config\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661805 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661780 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13a90b1a-e89d-4c50-87fe-5b08a0018d79-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661940 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661813 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13a90b1a-e89d-4c50-87fe-5b08a0018d79-config-out\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661940 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661832 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661940 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661882 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.661940 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661913 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.662128 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661937 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13a90b1a-e89d-4c50-87fe-5b08a0018d79-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.662128 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.661969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13a90b1a-e89d-4c50-87fe-5b08a0018d79-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.667656 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.667639 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:54:15.677746 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.677723 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad16142-0830-4c62-867c-33a4dd28d486" path="/var/lib/kubelet/pods/1ad16142-0830-4c62-867c-33a4dd28d486/volumes" Jul 23 21:54:15.762848 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.762816 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.762848 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.762850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763046 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.762867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13a90b1a-e89d-4c50-87fe-5b08a0018d79-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763046 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.762993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13a90b1a-e89d-4c50-87fe-5b08a0018d79-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763162 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763085 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763162 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763121 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763162 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-config-volume\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763300 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763268 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13a90b1a-e89d-4c50-87fe-5b08a0018d79-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763357 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763321 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gbrj\" (UniqueName: \"kubernetes.io/projected/13a90b1a-e89d-4c50-87fe-5b08a0018d79-kube-api-access-5gbrj\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-web-config\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13a90b1a-e89d-4c50-87fe-5b08a0018d79-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763397 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13a90b1a-e89d-4c50-87fe-5b08a0018d79-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763589 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763426 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13a90b1a-e89d-4c50-87fe-5b08a0018d79-config-out\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.763589 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.764070 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.763721 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13a90b1a-e89d-4c50-87fe-5b08a0018d79-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.764350 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.764325 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13a90b1a-e89d-4c50-87fe-5b08a0018d79-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766200 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.765954 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-config-volume\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766200 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.766129 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766200 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.766177 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766454 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.766225 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766454 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.766236 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766454 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.766271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.766843 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.766823 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13a90b1a-e89d-4c50-87fe-5b08a0018d79-web-config\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.767196 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.767179 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13a90b1a-e89d-4c50-87fe-5b08a0018d79-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.767828 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.767809 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13a90b1a-e89d-4c50-87fe-5b08a0018d79-config-out\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.773278 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.773260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gbrj\" (UniqueName: \"kubernetes.io/projected/13a90b1a-e89d-4c50-87fe-5b08a0018d79-kube-api-access-5gbrj\") pod \"alertmanager-main-0\" (UID: \"13a90b1a-e89d-4c50-87fe-5b08a0018d79\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:15.959909 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:15.959875 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 21:54:16.118067 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:16.118020 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 21:54:16.121396 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:54:16.121362 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a90b1a_e89d_4c50_87fe_5b08a0018d79.slice/crio-0bfa5ffed18dd2be9a11a4772c82f512ca37cdf8452b2ea58d9a7b7bcd8d4d3e WatchSource:0}: Error finding container 0bfa5ffed18dd2be9a11a4772c82f512ca37cdf8452b2ea58d9a7b7bcd8d4d3e: Status 404 returned error can't find the container with id 0bfa5ffed18dd2be9a11a4772c82f512ca37cdf8452b2ea58d9a7b7bcd8d4d3e Jul 23 21:54:16.577461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:16.577427 2579 generic.go:356] "Generic (PLEG): container finished" podID="13a90b1a-e89d-4c50-87fe-5b08a0018d79" containerID="e349fd08b4431bb9ba6a86cc42e3aecc070299bdc4aaee24c60f24b88296d66b" exitCode=0 Jul 23 21:54:16.577841 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:16.577510 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerDied","Data":"e349fd08b4431bb9ba6a86cc42e3aecc070299bdc4aaee24c60f24b88296d66b"} Jul 23 21:54:16.577841 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:16.577576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"0bfa5ffed18dd2be9a11a4772c82f512ca37cdf8452b2ea58d9a7b7bcd8d4d3e"} Jul 23 21:54:17.585554 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.585516 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"47c216efd1cc9607621a2f59dd495f081dd17b40656027b64a9e97a1abd78c62"} Jul 23 21:54:17.585898 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.585609 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"10b3e64edc0f4d52dc3425d5f5d4008b289ef247579b712fb60c539986881b1d"} Jul 23 21:54:17.585898 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.585640 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"9b0bc7613bc451b000120989938e851c531646b4d8c907ee8c2d9b9ec50cac3f"} Jul 23 21:54:17.585898 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.585664 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"5c91d8d5ca03541c4bf2269346a7e47757b35d19daf8ef1dde8e2a82c6010e69"} Jul 23 21:54:17.585898 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.585687 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"fd8bed2043185cf6e4c61c6920c629f2d3fd227d5b68bee82693cdb5234c85b1"} Jul 23 21:54:17.585898 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.585714 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13a90b1a-e89d-4c50-87fe-5b08a0018d79","Type":"ContainerStarted","Data":"ac2290994142d2033abcfea4e0b32d5e030e23356638545f29de8844a9868d6b"} Jul 23 21:54:17.590535 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.590509 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr"] Jul 23 21:54:17.594161 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.594140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.596719 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.596699 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 21:54:17.596985 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.596962 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 21:54:17.597138 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.597019 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ngnvc\"" Jul 23 21:54:17.597138 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.597043 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 21:54:17.597335 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.597319 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 21:54:17.597424 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.597360 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 21:54:17.601662 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.601641 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 21:54:17.609835 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.609816 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr"] Jul 23 21:54:17.619542 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.619492 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.603456876 podStartE2EDuration="2.619479063s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.022187ms" isStatefulPod=false podCreationTimestamp="2026-07-23 21:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:54:17.617137316 +0000 UTC m=+170.376208329" watchObservedRunningTime="2026-07-23 21:54:17.619479063 +0000 UTC m=+170.378550073" Jul 23 21:54:17.678557 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678527 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.678693 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678566 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7442\" (UniqueName: \"kubernetes.io/projected/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-kube-api-access-z7442\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.678693 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.678811 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678782 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-telemeter-client-tls\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.678873 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678855 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-metrics-client-ca\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.678930 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-federate-client-tls\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.678983 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.678923 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-secret-telemeter-client\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.679037 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.679026 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-serving-certs-ca-bundle\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.779767 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-metrics-client-ca\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.779767 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-federate-client-tls\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779790 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-secret-telemeter-client\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779819 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-serving-certs-ca-bundle\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7442\" (UniqueName: \"kubernetes.io/projected/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-kube-api-access-z7442\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779896 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.779925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-telemeter-client-tls\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780644 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.780592 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-metrics-client-ca\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.780644 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.780608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-serving-certs-ca-bundle\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.781168 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.781145 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.782407 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.782383 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-federate-client-tls\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.782537 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.782513 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.782759 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.782740 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-secret-telemeter-client\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.782823 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.782774 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-telemeter-client-tls\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.790138 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.790118 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7442\" (UniqueName: \"kubernetes.io/projected/f4e909e3-d953-42c9-8d6a-c09c5bdb3060-kube-api-access-z7442\") pod \"telemeter-client-7b69d45f7b-lnwsr\" (UID: \"f4e909e3-d953-42c9-8d6a-c09c5bdb3060\") " pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:17.905605 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:17.905562 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" Jul 23 21:54:18.049844 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:18.049812 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr"] Jul 23 21:54:18.591474 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:18.591435 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" event={"ID":"f4e909e3-d953-42c9-8d6a-c09c5bdb3060","Type":"ContainerStarted","Data":"7807c4af312e0888b17b27e142b02f883441f3e0fe440948121386e3bbde03df"} Jul 23 21:54:20.602477 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:20.602447 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" event={"ID":"f4e909e3-d953-42c9-8d6a-c09c5bdb3060","Type":"ContainerStarted","Data":"70c591aedf98550b35d407e0fd390a5c9665cfb2a0e826f48795e1c019eaedd6"} Jul 23 21:54:20.602886 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:20.602497 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" event={"ID":"f4e909e3-d953-42c9-8d6a-c09c5bdb3060","Type":"ContainerStarted","Data":"5fc0acd940be6a745ae79320ac7ae5920063e5e200caf70587accdfc28fe5541"} Jul 23 21:54:20.602886 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:20.602517 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" event={"ID":"f4e909e3-d953-42c9-8d6a-c09c5bdb3060","Type":"ContainerStarted","Data":"f834d55abca73fe7aba3b35f1ff1ca0382fce028b251bdf8040f318e177333dc"} Jul 23 21:54:20.631784 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:20.631729 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b69d45f7b-lnwsr" podStartSLOduration=1.89302177 podStartE2EDuration="3.631712404s" totalImagesPullingTime="1.738690634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:54:20.630928866 +0000 UTC m=+173.389999899" watchObservedRunningTime="2026-07-23 21:54:20.631712404 +0000 UTC m=+173.390783440" Jul 23 21:54:21.532958 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.532920 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75fc8cd9b-l2x79"] Jul 23 21:54:21.537200 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.537173 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.549851 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.549823 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75fc8cd9b-l2x79"] Jul 23 21:54:21.613889 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.613857 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-console-config\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.613889 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.613888 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-trusted-ca-bundle\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.614308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.613909 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-oauth-config\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.614308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.613932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-serving-cert\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.614308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.613965 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-service-ca\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.614308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.614009 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-oauth-serving-cert\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.614308 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.614038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scvdb\" (UniqueName: \"kubernetes.io/projected/0742f658-78ac-4eb6-878b-1259245ea545-kube-api-access-scvdb\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.715581 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.715545 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-console-config\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.715820 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.715608 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-trusted-ca-bundle\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.716252 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.716115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-oauth-config\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.716374 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.716343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-serving-cert\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.716439 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.716405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-service-ca\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.716547 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.716526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-oauth-serving-cert\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.716633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.716616 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scvdb\" (UniqueName: \"kubernetes.io/projected/0742f658-78ac-4eb6-878b-1259245ea545-kube-api-access-scvdb\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.716724 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.716703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-console-config\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.717690 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.717661 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-service-ca\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.719289 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.717855 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-oauth-serving-cert\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.719832 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.719807 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-serving-cert\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.719991 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.719968 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-oauth-config\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.720767 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.720745 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-trusted-ca-bundle\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.725188 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.725162 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scvdb\" (UniqueName: \"kubernetes.io/projected/0742f658-78ac-4eb6-878b-1259245ea545-kube-api-access-scvdb\") pod \"console-75fc8cd9b-l2x79\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.847563 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.847474 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:21.982851 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:21.982827 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75fc8cd9b-l2x79"] Jul 23 21:54:21.985502 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:54:21.985475 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0742f658_78ac_4eb6_878b_1259245ea545.slice/crio-612b74a138537257ca69afbddb550370d3076b7490402b4b801bbbd50bf8078b WatchSource:0}: Error finding container 612b74a138537257ca69afbddb550370d3076b7490402b4b801bbbd50bf8078b: Status 404 returned error can't find the container with id 612b74a138537257ca69afbddb550370d3076b7490402b4b801bbbd50bf8078b Jul 23 21:54:22.613480 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:22.613446 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75fc8cd9b-l2x79" event={"ID":"0742f658-78ac-4eb6-878b-1259245ea545","Type":"ContainerStarted","Data":"bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50"} Jul 23 21:54:22.613645 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:22.613492 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75fc8cd9b-l2x79" event={"ID":"0742f658-78ac-4eb6-878b-1259245ea545","Type":"ContainerStarted","Data":"612b74a138537257ca69afbddb550370d3076b7490402b4b801bbbd50bf8078b"} Jul 23 21:54:22.632715 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:22.632668 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75fc8cd9b-l2x79" podStartSLOduration=1.6326524610000002 podStartE2EDuration="1.632652461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:54:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:54:22.63116227 +0000 UTC m=+175.390233300" watchObservedRunningTime="2026-07-23 21:54:22.632652461 +0000 UTC m=+175.391723825" Jul 23 21:54:31.848574 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:31.848524 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:31.848574 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:31.848595 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:31.854179 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:31.854156 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:32.665714 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:32.665684 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:54:32.720085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:32.720016 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b9567c8f5-g9xhh"] Jul 23 21:54:57.743549 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:57.743495 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b9567c8f5-g9xhh" podUID="7ad871d9-5dd1-4658-8bb1-9bba843196a5" containerName="console" containerID="cri-o://104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45" gracePeriod=15 Jul 23 21:54:57.992490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:57.992408 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:54:58.131442 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131415 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-config\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131442 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131447 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-serving-cert\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131660 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131475 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqvl2\" (UniqueName: \"kubernetes.io/projected/7ad871d9-5dd1-4658-8bb1-9bba843196a5-kube-api-access-xqvl2\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131660 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131498 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-service-ca\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131660 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131579 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-oauth-serving-cert\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131660 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131650 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-trusted-ca-bundle\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131868 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131675 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-oauth-config\") pod \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\" (UID: \"7ad871d9-5dd1-4658-8bb1-9bba843196a5\") " Jul 23 21:54:58.131955 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.131872 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-config" (OuterVolumeSpecName: "console-config") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:54:58.132074 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.132016 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-service-ca" (OuterVolumeSpecName: "service-ca") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:54:58.132074 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.132037 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.132202 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.132133 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:54:58.132241 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.132216 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:54:58.133679 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.133647 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad871d9-5dd1-4658-8bb1-9bba843196a5-kube-api-access-xqvl2" (OuterVolumeSpecName: "kube-api-access-xqvl2") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "kube-api-access-xqvl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:54:58.133774 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.133684 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:58.133774 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.133699 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7ad871d9-5dd1-4658-8bb1-9bba843196a5" (UID: "7ad871d9-5dd1-4658-8bb1-9bba843196a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:54:58.233344 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.233308 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.233344 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.233337 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqvl2\" (UniqueName: \"kubernetes.io/projected/7ad871d9-5dd1-4658-8bb1-9bba843196a5-kube-api-access-xqvl2\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.233538 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.233352 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-service-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.233538 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.233363 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-oauth-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.233538 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.233374 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ad871d9-5dd1-4658-8bb1-9bba843196a5-trusted-ca-bundle\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.233538 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.233386 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7ad871d9-5dd1-4658-8bb1-9bba843196a5-console-oauth-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:54:58.774780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.774748 2579 generic.go:356] "Generic (PLEG): container finished" podID="7ad871d9-5dd1-4658-8bb1-9bba843196a5" containerID="104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45" exitCode=0 Jul 23 21:54:58.774780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.774784 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9567c8f5-g9xhh" event={"ID":"7ad871d9-5dd1-4658-8bb1-9bba843196a5","Type":"ContainerDied","Data":"104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45"} Jul 23 21:54:58.775353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.774798 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b9567c8f5-g9xhh" Jul 23 21:54:58.775353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.774817 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b9567c8f5-g9xhh" event={"ID":"7ad871d9-5dd1-4658-8bb1-9bba843196a5","Type":"ContainerDied","Data":"c709ed46b60ef60f2c45a274f3668e0660281fb01b4feea95e7c7572528d54e4"} Jul 23 21:54:58.775353 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.774836 2579 scope.go:117] "RemoveContainer" containerID="104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45" Jul 23 21:54:58.783674 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.783658 2579 scope.go:117] "RemoveContainer" containerID="104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45" Jul 23 21:54:58.783909 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:54:58.783894 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45\": container with ID starting with 104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45 not found: ID does not exist" containerID="104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45" Jul 23 21:54:58.783956 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.783918 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45"} err="failed to get container status \"104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45\": rpc error: code = NotFound desc = could not find container \"104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45\": container with ID starting with 104031a8de866d3329dad4733daa0417b1865d30e067564ae953be14f9ac5c45 not found: ID does not exist" Jul 23 21:54:58.804325 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.804296 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b9567c8f5-g9xhh"] Jul 23 21:54:58.817181 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:58.817158 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b9567c8f5-g9xhh"] Jul 23 21:54:59.678040 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:54:59.678006 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ad871d9-5dd1-4658-8bb1-9bba843196a5" path="/var/lib/kubelet/pods/7ad871d9-5dd1-4658-8bb1-9bba843196a5/volumes" Jul 23 21:55:33.136633 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.136598 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bd8698999-qqh5p"] Jul 23 21:55:33.137487 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.137457 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ad871d9-5dd1-4658-8bb1-9bba843196a5" containerName="console" Jul 23 21:55:33.137614 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.137568 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ad871d9-5dd1-4658-8bb1-9bba843196a5" containerName="console" Jul 23 21:55:33.137773 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.137755 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ad871d9-5dd1-4658-8bb1-9bba843196a5" containerName="console" Jul 23 21:55:33.142529 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.142507 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.153586 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.153561 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bd8698999-qqh5p"] Jul 23 21:55:33.223438 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223401 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-trusted-ca-bundle\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.223601 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223461 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-oauth-serving-cert\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.223601 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223513 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-console-config\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.223601 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvm4n\" (UniqueName: \"kubernetes.io/projected/6a759735-72a3-4679-b9d1-794aa70bad6c-kube-api-access-mvm4n\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.223601 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223551 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-serving-cert\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.223601 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-oauth-config\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.223759 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.223606 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-service-ca\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.324774 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324741 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-console-config\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.324774 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324773 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvm4n\" (UniqueName: \"kubernetes.io/projected/6a759735-72a3-4679-b9d1-794aa70bad6c-kube-api-access-mvm4n\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325030 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324792 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-serving-cert\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325030 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324812 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-oauth-config\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325030 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-service-ca\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325030 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324890 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-trusted-ca-bundle\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325030 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.324943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-oauth-serving-cert\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325736 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.325693 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-service-ca\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325859 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.325740 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-oauth-serving-cert\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325859 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.325743 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-trusted-ca-bundle\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.325859 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.325793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-console-config\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.327369 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.327351 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-serving-cert\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.327460 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.327415 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-oauth-config\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.334141 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.334112 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvm4n\" (UniqueName: \"kubernetes.io/projected/6a759735-72a3-4679-b9d1-794aa70bad6c-kube-api-access-mvm4n\") pod \"console-7bd8698999-qqh5p\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.454128 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.454011 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:33.595978 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.595954 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bd8698999-qqh5p"] Jul 23 21:55:33.597939 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:55:33.597911 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a759735_72a3_4679_b9d1_794aa70bad6c.slice/crio-d320e0c45bc74b958d6aa57b10c83a4f0edac3c9684f388e0116c4e8feee164c WatchSource:0}: Error finding container d320e0c45bc74b958d6aa57b10c83a4f0edac3c9684f388e0116c4e8feee164c: Status 404 returned error can't find the container with id d320e0c45bc74b958d6aa57b10c83a4f0edac3c9684f388e0116c4e8feee164c Jul 23 21:55:33.932570 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.932535 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd8698999-qqh5p" event={"ID":"6a759735-72a3-4679-b9d1-794aa70bad6c","Type":"ContainerStarted","Data":"747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b"} Jul 23 21:55:33.932736 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:33.932584 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd8698999-qqh5p" event={"ID":"6a759735-72a3-4679-b9d1-794aa70bad6c","Type":"ContainerStarted","Data":"d320e0c45bc74b958d6aa57b10c83a4f0edac3c9684f388e0116c4e8feee164c"} Jul 23 21:55:43.455129 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:43.455082 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:43.455547 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:43.455158 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:43.459843 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:43.459819 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:43.479091 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:43.479022 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bd8698999-qqh5p" podStartSLOduration=10.479001098 podStartE2EDuration="10.479001098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:55:33.9502893 +0000 UTC m=+246.709360311" watchObservedRunningTime="2026-07-23 21:55:43.479001098 +0000 UTC m=+256.238072111" Jul 23 21:55:43.980137 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:43.980108 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 21:55:44.027911 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:55:44.027879 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75fc8cd9b-l2x79"] Jul 23 21:56:09.055144 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.055100 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75fc8cd9b-l2x79" podUID="0742f658-78ac-4eb6-878b-1259245ea545" containerName="console" containerID="cri-o://bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50" gracePeriod=15 Jul 23 21:56:09.314834 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.314778 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:56:09.453600 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453561 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-oauth-serving-cert\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.453790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453628 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-serving-cert\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.453790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453656 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-service-ca\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.453790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453682 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-console-config\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.453790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453706 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scvdb\" (UniqueName: \"kubernetes.io/projected/0742f658-78ac-4eb6-878b-1259245ea545-kube-api-access-scvdb\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.453790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453748 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-trusted-ca-bundle\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.453790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.453788 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-oauth-config\") pod \"0742f658-78ac-4eb6-878b-1259245ea545\" (UID: \"0742f658-78ac-4eb6-878b-1259245ea545\") " Jul 23 21:56:09.454113 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.454017 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:56:09.454205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.454178 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-console-config" (OuterVolumeSpecName: "console-config") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:56:09.454260 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.454200 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-service-ca" (OuterVolumeSpecName: "service-ca") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:56:09.454303 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.454281 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:56:09.455934 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.455899 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:56:09.456039 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.455951 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:56:09.456039 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.456009 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0742f658-78ac-4eb6-878b-1259245ea545-kube-api-access-scvdb" (OuterVolumeSpecName: "kube-api-access-scvdb") pod "0742f658-78ac-4eb6-878b-1259245ea545" (UID: "0742f658-78ac-4eb6-878b-1259245ea545"). InnerVolumeSpecName "kube-api-access-scvdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:56:09.554731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554698 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-oauth-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:09.554731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554724 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:09.554731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554733 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-service-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:09.554731 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554741 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-console-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:09.554979 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554749 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scvdb\" (UniqueName: \"kubernetes.io/projected/0742f658-78ac-4eb6-878b-1259245ea545-kube-api-access-scvdb\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:09.554979 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554759 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0742f658-78ac-4eb6-878b-1259245ea545-trusted-ca-bundle\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:09.554979 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:09.554767 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0742f658-78ac-4eb6-878b-1259245ea545-console-oauth-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:56:10.089532 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.089445 2579 generic.go:356] "Generic (PLEG): container finished" podID="0742f658-78ac-4eb6-878b-1259245ea545" containerID="bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50" exitCode=0 Jul 23 21:56:10.089532 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.089505 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75fc8cd9b-l2x79" Jul 23 21:56:10.089989 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.089508 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75fc8cd9b-l2x79" event={"ID":"0742f658-78ac-4eb6-878b-1259245ea545","Type":"ContainerDied","Data":"bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50"} Jul 23 21:56:10.089989 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.089648 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75fc8cd9b-l2x79" event={"ID":"0742f658-78ac-4eb6-878b-1259245ea545","Type":"ContainerDied","Data":"612b74a138537257ca69afbddb550370d3076b7490402b4b801bbbd50bf8078b"} Jul 23 21:56:10.089989 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.089667 2579 scope.go:117] "RemoveContainer" containerID="bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50" Jul 23 21:56:10.097984 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.097964 2579 scope.go:117] "RemoveContainer" containerID="bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50" Jul 23 21:56:10.098243 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:56:10.098220 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50\": container with ID starting with bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50 not found: ID does not exist" containerID="bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50" Jul 23 21:56:10.098314 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.098251 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50"} err="failed to get container status \"bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50\": rpc error: code = NotFound desc = could not find container \"bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50\": container with ID starting with bc3957c30b806ddd876db4e9f320540c450c9fc4d13411835404b35b7e27de50 not found: ID does not exist" Jul 23 21:56:10.116269 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.116242 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75fc8cd9b-l2x79"] Jul 23 21:56:10.120645 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:10.120615 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75fc8cd9b-l2x79"] Jul 23 21:56:11.677757 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:11.677723 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0742f658-78ac-4eb6-878b-1259245ea545" path="/var/lib/kubelet/pods/0742f658-78ac-4eb6-878b-1259245ea545/volumes" Jul 23 21:56:27.583068 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:27.583025 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 21:56:27.587831 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:27.587810 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 21:56:27.589579 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:27.589559 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:56:27.598216 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:27.598199 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 21:56:27.599386 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:27.599369 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 21:56:52.412921 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.412880 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk"] Jul 23 21:56:52.419077 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.413735 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0742f658-78ac-4eb6-878b-1259245ea545" containerName="console" Jul 23 21:56:52.419077 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.413755 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0742f658-78ac-4eb6-878b-1259245ea545" containerName="console" Jul 23 21:56:52.419077 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.413927 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0742f658-78ac-4eb6-878b-1259245ea545" containerName="console" Jul 23 21:56:52.419241 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.419099 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.421903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.421873 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:56:52.422017 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.421958 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 21:56:52.422085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.422021 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hlbs5\"" Jul 23 21:56:52.430817 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.430795 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk"] Jul 23 21:56:52.546848 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.546815 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/699253b7-7aec-4dab-a408-800d25287914-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2dnlk\" (UID: \"699253b7-7aec-4dab-a408-800d25287914\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.547028 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.546865 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j84vs\" (UniqueName: \"kubernetes.io/projected/699253b7-7aec-4dab-a408-800d25287914-kube-api-access-j84vs\") pod \"cert-manager-operator-controller-manager-55c444bf86-2dnlk\" (UID: \"699253b7-7aec-4dab-a408-800d25287914\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.648286 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.648245 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j84vs\" (UniqueName: \"kubernetes.io/projected/699253b7-7aec-4dab-a408-800d25287914-kube-api-access-j84vs\") pod \"cert-manager-operator-controller-manager-55c444bf86-2dnlk\" (UID: \"699253b7-7aec-4dab-a408-800d25287914\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.648478 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.648365 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/699253b7-7aec-4dab-a408-800d25287914-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2dnlk\" (UID: \"699253b7-7aec-4dab-a408-800d25287914\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.648737 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.648715 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/699253b7-7aec-4dab-a408-800d25287914-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-2dnlk\" (UID: \"699253b7-7aec-4dab-a408-800d25287914\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.659780 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.659751 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j84vs\" (UniqueName: \"kubernetes.io/projected/699253b7-7aec-4dab-a408-800d25287914-kube-api-access-j84vs\") pod \"cert-manager-operator-controller-manager-55c444bf86-2dnlk\" (UID: \"699253b7-7aec-4dab-a408-800d25287914\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.729326 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.729255 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" Jul 23 21:56:52.871380 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.871352 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk"] Jul 23 21:56:52.872958 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:56:52.872935 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod699253b7_7aec_4dab_a408_800d25287914.slice/crio-f7f8604d6233022c81290ba75591873d5255c6ea3b925c923a3cce470c40398f WatchSource:0}: Error finding container f7f8604d6233022c81290ba75591873d5255c6ea3b925c923a3cce470c40398f: Status 404 returned error can't find the container with id f7f8604d6233022c81290ba75591873d5255c6ea3b925c923a3cce470c40398f Jul 23 21:56:52.875761 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:52.875745 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 21:56:53.283307 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:53.283270 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" event={"ID":"699253b7-7aec-4dab-a408-800d25287914","Type":"ContainerStarted","Data":"f7f8604d6233022c81290ba75591873d5255c6ea3b925c923a3cce470c40398f"} Jul 23 21:56:56.301390 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:56.301355 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" event={"ID":"699253b7-7aec-4dab-a408-800d25287914","Type":"ContainerStarted","Data":"b43b1b6ddf7c9cdc447602ea7b3b0e977308aa7ae886f44364fea6454fe1a6f1"} Jul 23 21:56:56.325922 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:56.325864 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-2dnlk" podStartSLOduration=1.403931359 podStartE2EDuration="4.325845569s" totalImagesPullingTime="2.92191421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:56:56.323496993 +0000 UTC m=+329.082568004" watchObservedRunningTime="2026-07-23 21:56:56.325845569 +0000 UTC m=+329.084916579" Jul 23 21:56:59.231302 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.231265 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-87v6z"] Jul 23 21:56:59.234769 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.234750 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.237241 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.237219 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 21:56:59.238431 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.238414 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2h4ds\"" Jul 23 21:56:59.238531 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.238443 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 21:56:59.244546 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.244521 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-87v6z"] Jul 23 21:56:59.308264 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.308228 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh7q4\" (UniqueName: \"kubernetes.io/projected/3e202e4c-77a5-4b70-b0f7-8c903144d3a6-kube-api-access-rh7q4\") pod \"cert-manager-cainjector-559cdd8c87-87v6z\" (UID: \"3e202e4c-77a5-4b70-b0f7-8c903144d3a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.308407 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.308292 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e202e4c-77a5-4b70-b0f7-8c903144d3a6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-87v6z\" (UID: \"3e202e4c-77a5-4b70-b0f7-8c903144d3a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.409398 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.409363 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh7q4\" (UniqueName: \"kubernetes.io/projected/3e202e4c-77a5-4b70-b0f7-8c903144d3a6-kube-api-access-rh7q4\") pod \"cert-manager-cainjector-559cdd8c87-87v6z\" (UID: \"3e202e4c-77a5-4b70-b0f7-8c903144d3a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.409577 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.409447 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e202e4c-77a5-4b70-b0f7-8c903144d3a6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-87v6z\" (UID: \"3e202e4c-77a5-4b70-b0f7-8c903144d3a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.417491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.417458 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e202e4c-77a5-4b70-b0f7-8c903144d3a6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-87v6z\" (UID: \"3e202e4c-77a5-4b70-b0f7-8c903144d3a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.417681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.417659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh7q4\" (UniqueName: \"kubernetes.io/projected/3e202e4c-77a5-4b70-b0f7-8c903144d3a6-kube-api-access-rh7q4\") pod \"cert-manager-cainjector-559cdd8c87-87v6z\" (UID: \"3e202e4c-77a5-4b70-b0f7-8c903144d3a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.549114 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.549017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" Jul 23 21:56:59.680730 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:56:59.680705 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-87v6z"] Jul 23 21:56:59.683222 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:56:59.683191 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e202e4c_77a5_4b70_b0f7_8c903144d3a6.slice/crio-57be58dfeeeba6184e9362ab0d2082275bb5334a883010258cfcc4a7062ac8cf WatchSource:0}: Error finding container 57be58dfeeeba6184e9362ab0d2082275bb5334a883010258cfcc4a7062ac8cf: Status 404 returned error can't find the container with id 57be58dfeeeba6184e9362ab0d2082275bb5334a883010258cfcc4a7062ac8cf Jul 23 21:57:00.320895 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:00.320860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" event={"ID":"3e202e4c-77a5-4b70-b0f7-8c903144d3a6","Type":"ContainerStarted","Data":"57be58dfeeeba6184e9362ab0d2082275bb5334a883010258cfcc4a7062ac8cf"} Jul 23 21:57:03.337270 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:03.337236 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" event={"ID":"3e202e4c-77a5-4b70-b0f7-8c903144d3a6","Type":"ContainerStarted","Data":"667c98732957450134ab35e7a14a96d4f7f625d6c1ccf869fe30c61ab6b0740d"} Jul 23 21:57:03.362413 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:03.362363 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-87v6z" podStartSLOduration=1.6840582290000001 podStartE2EDuration="4.362348719s" totalImagesPullingTime="2.67829049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:03.360012202 +0000 UTC m=+336.119083232" watchObservedRunningTime="2026-07-23 21:57:03.362348719 +0000 UTC m=+336.121419730" Jul 23 21:57:14.697388 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.697354 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm"] Jul 23 21:57:14.700755 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.700739 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:14.703301 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.703283 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-frw8b\"" Jul 23 21:57:14.703406 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.703283 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:57:14.704531 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.704515 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 21:57:14.709008 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.708989 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm"] Jul 23 21:57:14.747267 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.747237 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e77d809-9469-40ac-ac6d-150d9ec2ef38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-926mm\" (UID: \"1e77d809-9469-40ac-ac6d-150d9ec2ef38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:14.747438 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.747272 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgl5c\" (UniqueName: \"kubernetes.io/projected/1e77d809-9469-40ac-ac6d-150d9ec2ef38-kube-api-access-vgl5c\") pod \"openshift-lws-operator-fd8ccff4c-926mm\" (UID: \"1e77d809-9469-40ac-ac6d-150d9ec2ef38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:14.847855 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.847817 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e77d809-9469-40ac-ac6d-150d9ec2ef38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-926mm\" (UID: \"1e77d809-9469-40ac-ac6d-150d9ec2ef38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:14.848002 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.847859 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgl5c\" (UniqueName: \"kubernetes.io/projected/1e77d809-9469-40ac-ac6d-150d9ec2ef38-kube-api-access-vgl5c\") pod \"openshift-lws-operator-fd8ccff4c-926mm\" (UID: \"1e77d809-9469-40ac-ac6d-150d9ec2ef38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:14.848284 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.848265 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e77d809-9469-40ac-ac6d-150d9ec2ef38-tmp\") pod \"openshift-lws-operator-fd8ccff4c-926mm\" (UID: \"1e77d809-9469-40ac-ac6d-150d9ec2ef38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:14.856603 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:14.856582 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgl5c\" (UniqueName: \"kubernetes.io/projected/1e77d809-9469-40ac-ac6d-150d9ec2ef38-kube-api-access-vgl5c\") pod \"openshift-lws-operator-fd8ccff4c-926mm\" (UID: \"1e77d809-9469-40ac-ac6d-150d9ec2ef38\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:15.010244 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:15.010161 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" Jul 23 21:57:15.141099 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:15.141074 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm"] Jul 23 21:57:15.143194 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:57:15.143168 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e77d809_9469_40ac_ac6d_150d9ec2ef38.slice/crio-51ba64ca6728114935b85e2d90eff6c9fb91058d31f7edbb3f614161c278a9cf WatchSource:0}: Error finding container 51ba64ca6728114935b85e2d90eff6c9fb91058d31f7edbb3f614161c278a9cf: Status 404 returned error can't find the container with id 51ba64ca6728114935b85e2d90eff6c9fb91058d31f7edbb3f614161c278a9cf Jul 23 21:57:15.392461 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:15.392428 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" event={"ID":"1e77d809-9469-40ac-ac6d-150d9ec2ef38","Type":"ContainerStarted","Data":"51ba64ca6728114935b85e2d90eff6c9fb91058d31f7edbb3f614161c278a9cf"} Jul 23 21:57:18.409228 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:18.409193 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" event={"ID":"1e77d809-9469-40ac-ac6d-150d9ec2ef38","Type":"ContainerStarted","Data":"52492cdb77f5bf8609d7e0ae4c40565a29ab094e62e507536750d42f9da49664"} Jul 23 21:57:18.427997 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:18.427945 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-926mm" podStartSLOduration=1.908575408 podStartE2EDuration="4.42793016s" totalImagesPullingTime="2.519354752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:18.425266873 +0000 UTC m=+351.184337948" watchObservedRunningTime="2026-07-23 21:57:18.42793016 +0000 UTC m=+351.187001171" Jul 23 21:57:33.855603 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.855569 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4"] Jul 23 21:57:33.860281 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.860261 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:33.865010 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.864983 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 21:57:33.865348 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.865332 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 21:57:33.865420 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.865373 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 21:57:33.865420 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.865411 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qjdhj\"" Jul 23 21:57:33.868949 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.868933 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 21:57:33.889530 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.889497 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4"] Jul 23 21:57:33.920688 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.920655 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac36272a-c11d-4741-bd75-978974926cd7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:33.920688 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.920691 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac36272a-c11d-4741-bd75-978974926cd7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:33.920902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.920732 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac36272a-c11d-4741-bd75-978974926cd7-tmp\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:33.920902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.920778 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac36272a-c11d-4741-bd75-978974926cd7-manifests\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:33.920902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:33.920869 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqjb\" (UniqueName: \"kubernetes.io/projected/ac36272a-c11d-4741-bd75-978974926cd7-kube-api-access-mwqjb\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.021565 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.021532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqjb\" (UniqueName: \"kubernetes.io/projected/ac36272a-c11d-4741-bd75-978974926cd7-kube-api-access-mwqjb\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.021738 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.021613 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac36272a-c11d-4741-bd75-978974926cd7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.021797 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.021769 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac36272a-c11d-4741-bd75-978974926cd7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.021891 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.021874 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac36272a-c11d-4741-bd75-978974926cd7-tmp\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.021932 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.021915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac36272a-c11d-4741-bd75-978974926cd7-manifests\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.022307 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.022280 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ac36272a-c11d-4741-bd75-978974926cd7-manifests\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.024087 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.024045 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac36272a-c11d-4741-bd75-978974926cd7-tmp\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.024201 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.024187 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ac36272a-c11d-4741-bd75-978974926cd7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.024288 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.024273 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ac36272a-c11d-4741-bd75-978974926cd7-webhook-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.032720 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.032691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqjb\" (UniqueName: \"kubernetes.io/projected/ac36272a-c11d-4741-bd75-978974926cd7-kube-api-access-mwqjb\") pod \"opendatahub-operator-controller-manager-6864c64679-2snh4\" (UID: \"ac36272a-c11d-4741-bd75-978974926cd7\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.171119 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.171030 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:34.311321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.311286 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4"] Jul 23 21:57:34.316231 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:57:34.316193 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac36272a_c11d_4741_bd75_978974926cd7.slice/crio-000befd5721f31cd3745a982a77d803957aace6768fab7902af56983776c9002 WatchSource:0}: Error finding container 000befd5721f31cd3745a982a77d803957aace6768fab7902af56983776c9002: Status 404 returned error can't find the container with id 000befd5721f31cd3745a982a77d803957aace6768fab7902af56983776c9002 Jul 23 21:57:34.483119 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:34.483021 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" event={"ID":"ac36272a-c11d-4741-bd75-978974926cd7","Type":"ContainerStarted","Data":"000befd5721f31cd3745a982a77d803957aace6768fab7902af56983776c9002"} Jul 23 21:57:37.502493 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:37.502414 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" event={"ID":"ac36272a-c11d-4741-bd75-978974926cd7","Type":"ContainerStarted","Data":"6df3d05ec1efd4e2cd6e5b3186ed70cc5776912666e9104a18cb1978f0040fea"} Jul 23 21:57:38.508420 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:38.508386 2579 generic.go:356] "Generic (PLEG): container finished" podID="ac36272a-c11d-4741-bd75-978974926cd7" containerID="6df3d05ec1efd4e2cd6e5b3186ed70cc5776912666e9104a18cb1978f0040fea" exitCode=0 Jul 23 21:57:38.508838 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:38.508468 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" event={"ID":"ac36272a-c11d-4741-bd75-978974926cd7","Type":"ContainerDied","Data":"6df3d05ec1efd4e2cd6e5b3186ed70cc5776912666e9104a18cb1978f0040fea"} Jul 23 21:57:39.514926 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:39.514891 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" event={"ID":"ac36272a-c11d-4741-bd75-978974926cd7","Type":"ContainerStarted","Data":"5a6f5723c7e0cb952f6c38d11fb094161c70133e806b9aa10246773ec2d3b3df"} Jul 23 21:57:39.515420 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:39.515018 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:39.541097 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:39.541027 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" podStartSLOduration=2.225476809 podStartE2EDuration="6.541009685s" totalImagesPullingTime="3.247173846s" totalInitContainerRuntime="1.06835903s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:39.539822633 +0000 UTC m=+372.298893655" watchObservedRunningTime="2026-07-23 21:57:39.541009685 +0000 UTC m=+372.300080695" Jul 23 21:57:50.521715 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:50.521685 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-2snh4" Jul 23 21:57:59.440497 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.440460 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87"] Jul 23 21:57:59.444022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.443996 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.446627 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.446602 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 23 21:57:59.446627 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.446608 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 23 21:57:59.447798 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.447780 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r9c92\"" Jul 23 21:57:59.456989 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.456964 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87"] Jul 23 21:57:59.543366 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.543338 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzqj9\" (UniqueName: \"kubernetes.io/projected/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-kube-api-access-nzqj9\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.543534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.543378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-tls-certs\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.543534 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.543452 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-tmp\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.566291 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.566259 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wft6m"] Jul 23 21:57:59.569655 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.569634 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.572884 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.572867 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2gqhz\"" Jul 23 21:57:59.573207 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.573185 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 23 21:57:59.582426 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.582405 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wft6m"] Jul 23 21:57:59.645046 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.645010 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzqj9\" (UniqueName: \"kubernetes.io/projected/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-kube-api-access-nzqj9\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.645220 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.645093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-tls-certs\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.645220 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.645163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-tmp\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.647691 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.647660 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-tmp\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.647825 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.647760 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-tls-certs\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.653008 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.652987 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzqj9\" (UniqueName: \"kubernetes.io/projected/4fc52e39-8b1e-4ea4-81e9-32b3f97920e9-kube-api-access-nzqj9\") pod \"kube-auth-proxy-5b78d9f867-6bs87\" (UID: \"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9\") " pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.745929 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.745844 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3df4a877-bece-4904-a5df-d3b2abcdd064-cert\") pod \"odh-model-controller-7998c67dc4-wft6m\" (UID: \"3df4a877-bece-4904-a5df-d3b2abcdd064\") " pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.746129 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.745928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p55fp\" (UniqueName: \"kubernetes.io/projected/3df4a877-bece-4904-a5df-d3b2abcdd064-kube-api-access-p55fp\") pod \"odh-model-controller-7998c67dc4-wft6m\" (UID: \"3df4a877-bece-4904-a5df-d3b2abcdd064\") " pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.753792 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.753774 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" Jul 23 21:57:59.846971 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.846929 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3df4a877-bece-4904-a5df-d3b2abcdd064-cert\") pod \"odh-model-controller-7998c67dc4-wft6m\" (UID: \"3df4a877-bece-4904-a5df-d3b2abcdd064\") " pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.847160 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.847027 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p55fp\" (UniqueName: \"kubernetes.io/projected/3df4a877-bece-4904-a5df-d3b2abcdd064-kube-api-access-p55fp\") pod \"odh-model-controller-7998c67dc4-wft6m\" (UID: \"3df4a877-bece-4904-a5df-d3b2abcdd064\") " pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.849529 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.849497 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3df4a877-bece-4904-a5df-d3b2abcdd064-cert\") pod \"odh-model-controller-7998c67dc4-wft6m\" (UID: \"3df4a877-bece-4904-a5df-d3b2abcdd064\") " pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.857107 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.857078 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p55fp\" (UniqueName: \"kubernetes.io/projected/3df4a877-bece-4904-a5df-d3b2abcdd064-kube-api-access-p55fp\") pod \"odh-model-controller-7998c67dc4-wft6m\" (UID: \"3df4a877-bece-4904-a5df-d3b2abcdd064\") " pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.879990 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.879958 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:57:59.887495 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:57:59.887460 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87"] Jul 23 21:57:59.890907 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:57:59.890882 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc52e39_8b1e_4ea4_81e9_32b3f97920e9.slice/crio-f7b414d739fd6033c9d2a814081761cd9014acc95fc8d9717f6e3e5df5957821 WatchSource:0}: Error finding container f7b414d739fd6033c9d2a814081761cd9014acc95fc8d9717f6e3e5df5957821: Status 404 returned error can't find the container with id f7b414d739fd6033c9d2a814081761cd9014acc95fc8d9717f6e3e5df5957821 Jul 23 21:58:00.229538 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.229368 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wft6m"] Jul 23 21:58:00.231703 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:00.231674 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df4a877_bece_4904_a5df_d3b2abcdd064.slice/crio-815d0f77eed25b7782e829a3b9ca941f2f9f6cc69fe45e0ec2c60b48e2211c45 WatchSource:0}: Error finding container 815d0f77eed25b7782e829a3b9ca941f2f9f6cc69fe45e0ec2c60b48e2211c45: Status 404 returned error can't find the container with id 815d0f77eed25b7782e829a3b9ca941f2f9f6cc69fe45e0ec2c60b48e2211c45 Jul 23 21:58:00.288788 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.288754 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z96zp"] Jul 23 21:58:00.293409 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.293388 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.298851 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.298833 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qb7sr\"" Jul 23 21:58:00.304033 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.304011 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 23 21:58:00.324200 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.324172 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z96zp"] Jul 23 21:58:00.350893 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.350864 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.351024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.350904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgmb8\" (UniqueName: \"kubernetes.io/projected/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-kube-api-access-jgmb8\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.452420 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.452355 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.452420 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.452420 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgmb8\" (UniqueName: \"kubernetes.io/projected/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-kube-api-access-jgmb8\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.452976 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:00.452550 2579 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 23 21:58:00.452976 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:00.452670 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-tls-certs podName:7160cbc7-767d-41ca-a42e-c1409b3aa6a2 nodeName:}" failed. No retries permitted until 2026-07-23 21:58:00.952630093 +0000 UTC m=+393.711701098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-tls-certs") pod "model-serving-api-54d89bd5fc-z96zp" (UID: "7160cbc7-767d-41ca-a42e-c1409b3aa6a2") : secret "model-serving-api-tls" not found Jul 23 21:58:00.471684 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.471655 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgmb8\" (UniqueName: \"kubernetes.io/projected/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-kube-api-access-jgmb8\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.612210 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.612170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" event={"ID":"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9","Type":"ContainerStarted","Data":"f7b414d739fd6033c9d2a814081761cd9014acc95fc8d9717f6e3e5df5957821"} Jul 23 21:58:00.613485 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.613457 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" event={"ID":"3df4a877-bece-4904-a5df-d3b2abcdd064","Type":"ContainerStarted","Data":"815d0f77eed25b7782e829a3b9ca941f2f9f6cc69fe45e0ec2c60b48e2211c45"} Jul 23 21:58:00.906890 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.906807 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm"] Jul 23 21:58:00.914938 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.914529 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.918566 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.917721 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 21:58:00.918566 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.918078 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 21:58:00.918566 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.918253 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 21:58:00.918566 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.918438 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jth9l\"" Jul 23 21:58:00.920027 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.919823 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 21:58:00.939596 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.939514 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm"] Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958688 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958733 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-token\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958761 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958785 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwzvf\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-kube-api-access-cwzvf\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958821 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-local-certs\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958893 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-cacerts\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958917 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.959488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.958950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:00.962750 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:00.962703 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7160cbc7-767d-41ca-a42e-c1409b3aa6a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-z96zp\" (UID: \"7160cbc7-767d-41ca-a42e-c1409b3aa6a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-cacerts\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059724 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059762 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059847 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-token\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059872 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059897 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwzvf\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-kube-api-access-cwzvf\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.060085 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.059935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-local-certs\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.061771 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.061510 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.063279 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.063255 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.064033 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.064011 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-local-certs\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.065272 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.065231 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-cacerts\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.066343 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.066323 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.075423 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.075396 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-token\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.078766 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.078734 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwzvf\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-kube-api-access-cwzvf\") pod \"istiod-openshift-gateway-7574459656-kvsrm\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.206627 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.204934 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:01.233024 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.232998 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:01.402151 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.402045 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-z96zp"] Jul 23 21:58:01.406619 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:01.406580 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7160cbc7_767d_41ca_a42e_c1409b3aa6a2.slice/crio-63b3da6293a0c6180565c9a2ac7ddd730877ce171474c8cdc3bc7387c588b2a8 WatchSource:0}: Error finding container 63b3da6293a0c6180565c9a2ac7ddd730877ce171474c8cdc3bc7387c588b2a8: Status 404 returned error can't find the container with id 63b3da6293a0c6180565c9a2ac7ddd730877ce171474c8cdc3bc7387c588b2a8 Jul 23 21:58:01.434016 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.433951 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm"] Jul 23 21:58:01.436394 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:01.436360 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f4bfdc3_1e16_4b44_a1dc_7d12a5164d73.slice/crio-297487d3d8043deda2b4d8e4a7aa18a624342ada9391bad8aee1745f95939acb WatchSource:0}: Error finding container 297487d3d8043deda2b4d8e4a7aa18a624342ada9391bad8aee1745f95939acb: Status 404 returned error can't find the container with id 297487d3d8043deda2b4d8e4a7aa18a624342ada9391bad8aee1745f95939acb Jul 23 21:58:01.621206 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.621170 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" event={"ID":"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73","Type":"ContainerStarted","Data":"297487d3d8043deda2b4d8e4a7aa18a624342ada9391bad8aee1745f95939acb"} Jul 23 21:58:01.622625 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:01.622597 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" event={"ID":"7160cbc7-767d-41ca-a42e-c1409b3aa6a2","Type":"ContainerStarted","Data":"63b3da6293a0c6180565c9a2ac7ddd730877ce171474c8cdc3bc7387c588b2a8"} Jul 23 21:58:05.147649 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.147603 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 21:58:05.148040 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.147695 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 21:58:05.649153 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.649111 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" event={"ID":"4fc52e39-8b1e-4ea4-81e9-32b3f97920e9","Type":"ContainerStarted","Data":"274da9ef5413c838011439b88e53a6f91527c5c20604e0262d725a882cc892af"} Jul 23 21:58:05.650752 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.650718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" event={"ID":"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73","Type":"ContainerStarted","Data":"4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02"} Jul 23 21:58:05.650884 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.650769 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:05.652229 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.652205 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" event={"ID":"3df4a877-bece-4904-a5df-d3b2abcdd064","Type":"ContainerStarted","Data":"d5c357b5730ab137f0ab59336be39c66f9494569f5fd8f18cb58bcd01a64d519"} Jul 23 21:58:05.652334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.652253 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:58:05.653788 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.653771 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" event={"ID":"7160cbc7-767d-41ca-a42e-c1409b3aa6a2","Type":"ContainerStarted","Data":"18b3958625d52eb80f9ea5ffab05cdb157c08a80cafaf9417808778937d265b4"} Jul 23 21:58:05.653902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.653890 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:05.679646 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.679591 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b78d9f867-6bs87" podStartSLOduration=3.009262529 podStartE2EDuration="6.679571128s" totalImagesPullingTime="3.670308599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:05.678810482 +0000 UTC m=+398.437881498" watchObservedRunningTime="2026-07-23 21:58:05.679571128 +0000 UTC m=+398.438642158" Jul 23 21:58:05.714631 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.714575 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" podStartSLOduration=2.036614964 podStartE2EDuration="5.714558138s" totalImagesPullingTime="3.677943174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:05.713611058 +0000 UTC m=+398.472682069" watchObservedRunningTime="2026-07-23 21:58:05.714558138 +0000 UTC m=+398.473629148" Jul 23 21:58:05.749597 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.749540 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" podStartSLOduration=2.04154401 podStartE2EDuration="5.7495169s" totalImagesPullingTime="3.70797289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:05.747842678 +0000 UTC m=+398.506913717" watchObservedRunningTime="2026-07-23 21:58:05.7495169 +0000 UTC m=+398.508587893" Jul 23 21:58:05.776140 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:05.776088 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" podStartSLOduration=1.924680502 podStartE2EDuration="6.77604115s" totalImagesPullingTime="4.851360648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:05.773921182 +0000 UTC m=+398.532992192" watchObservedRunningTime="2026-07-23 21:58:05.77604115 +0000 UTC m=+398.535112160" Jul 23 21:58:06.660313 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:06.660277 2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-kvsrm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 21:58:06.660685 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:06.660332 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" podUID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 21:58:09.659763 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:09.659733 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:11.017747 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.017713 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qll9x"] Jul 23 21:58:11.026042 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.025703 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.030388 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.030299 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xpcpn\"" Jul 23 21:58:11.030683 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.030655 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 23 21:58:11.031274 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.031252 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v"] Jul 23 21:58:11.034205 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.034185 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.035116 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.034803 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qll9x"] Jul 23 21:58:11.037449 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.037425 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gzpzz\"" Jul 23 21:58:11.037680 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.037663 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 23 21:58:11.047481 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.047457 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v"] Jul 23 21:58:11.052209 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.052189 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/342c3d0c-6c49-4af9-9237-d37885218558-cert\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.052298 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.052243 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq264\" (UniqueName: \"kubernetes.io/projected/342c3d0c-6c49-4af9-9237-d37885218558-kube-api-access-bq264\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.153385 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.153353 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6jnf\" (UniqueName: \"kubernetes.io/projected/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-kube-api-access-l6jnf\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.153542 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.153403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/342c3d0c-6c49-4af9-9237-d37885218558-cert\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.153542 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.153455 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.153542 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:11.153493 2579 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 23 21:58:11.153542 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.153529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq264\" (UniqueName: \"kubernetes.io/projected/342c3d0c-6c49-4af9-9237-d37885218558-kube-api-access-bq264\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.153673 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:11.153555 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/342c3d0c-6c49-4af9-9237-d37885218558-cert podName:342c3d0c-6c49-4af9-9237-d37885218558 nodeName:}" failed. No retries permitted until 2026-07-23 21:58:11.653536114 +0000 UTC m=+404.412607103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/342c3d0c-6c49-4af9-9237-d37885218558-cert") pod "kserve-controller-manager-d6f4b65b8-qll9x" (UID: "342c3d0c-6c49-4af9-9237-d37885218558") : secret "kserve-webhook-server-cert" not found Jul 23 21:58:11.165554 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.165523 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq264\" (UniqueName: \"kubernetes.io/projected/342c3d0c-6c49-4af9-9237-d37885218558-kube-api-access-bq264\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.254494 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.254450 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.254677 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:11.254569 2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 23 21:58:11.254677 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.254601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6jnf\" (UniqueName: \"kubernetes.io/projected/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-kube-api-access-l6jnf\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.254677 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:11.254633 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-cert podName:589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3 nodeName:}" failed. No retries permitted until 2026-07-23 21:58:11.754617423 +0000 UTC m=+404.513688412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-cert") pod "llmisvc-controller-manager-777c6bd7d5-wkl4v" (UID: "589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3") : secret "llmisvc-webhook-server-cert" not found Jul 23 21:58:11.272592 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.272518 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6jnf\" (UniqueName: \"kubernetes.io/projected/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-kube-api-access-l6jnf\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.658439 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.658406 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/342c3d0c-6c49-4af9-9237-d37885218558-cert\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.660903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.660873 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/342c3d0c-6c49-4af9-9237-d37885218558-cert\") pod \"kserve-controller-manager-d6f4b65b8-qll9x\" (UID: \"342c3d0c-6c49-4af9-9237-d37885218558\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.759691 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.759646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.762240 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.762211 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-wkl4v\" (UID: \"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:11.940022 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.939949 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:11.947963 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:11.947934 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:12.107853 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:12.107820 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qll9x"] Jul 23 21:58:12.113997 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:12.113967 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342c3d0c_6c49_4af9_9237_d37885218558.slice/crio-8de11cef5f1ee110a31b471766abeb504dea187d58eea005a6472aa936c3d162 WatchSource:0}: Error finding container 8de11cef5f1ee110a31b471766abeb504dea187d58eea005a6472aa936c3d162: Status 404 returned error can't find the container with id 8de11cef5f1ee110a31b471766abeb504dea187d58eea005a6472aa936c3d162 Jul 23 21:58:12.149110 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:12.149088 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v"] Jul 23 21:58:12.151256 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:12.151226 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod589d90cb_0c8d_46bc_9c4d_7dbaa12b75e3.slice/crio-b44983fd7c7854b0595c57c3ae1150ae25a52b3d88c03724be6f766f0ae6806e WatchSource:0}: Error finding container b44983fd7c7854b0595c57c3ae1150ae25a52b3d88c03724be6f766f0ae6806e: Status 404 returned error can't find the container with id b44983fd7c7854b0595c57c3ae1150ae25a52b3d88c03724be6f766f0ae6806e Jul 23 21:58:12.703689 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:12.697858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" event={"ID":"342c3d0c-6c49-4af9-9237-d37885218558","Type":"ContainerStarted","Data":"8de11cef5f1ee110a31b471766abeb504dea187d58eea005a6472aa936c3d162"} Jul 23 21:58:12.703689 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:12.699265 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" event={"ID":"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3","Type":"ContainerStarted","Data":"b44983fd7c7854b0595c57c3ae1150ae25a52b3d88c03724be6f766f0ae6806e"} Jul 23 21:58:13.116188 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.116086 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm"] Jul 23 21:58:13.117872 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.116920 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" podUID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" containerName="discovery" containerID="cri-o://4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02" gracePeriod=30 Jul 23 21:58:13.554382 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.554354 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:13.680007 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.679929 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-dns-cert\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.680007 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.679992 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-local-certs\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.680230 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.680027 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-token\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.680230 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.680076 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-kubeconfig\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.680230 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.680106 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwzvf\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-kube-api-access-cwzvf\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.681673 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.681103 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-ca-configmap\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.681673 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.681161 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-cacerts\") pod \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\" (UID: \"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73\") " Jul 23 21:58:13.681953 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.681922 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:58:13.683275 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.683231 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-kube-api-access-cwzvf" (OuterVolumeSpecName: "kube-api-access-cwzvf") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "kube-api-access-cwzvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:58:13.683517 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.683489 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-local-certs" (OuterVolumeSpecName: "local-certs") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:58:13.684149 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.684107 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:58:13.685348 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.685318 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:58:13.685448 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.685378 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-cacerts" (OuterVolumeSpecName: "cacerts") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:58:13.685637 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.685610 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" (UID: "2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:58:13.707632 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.707603 2579 generic.go:356] "Generic (PLEG): container finished" podID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" containerID="4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02" exitCode=0 Jul 23 21:58:13.707772 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.707650 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" event={"ID":"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73","Type":"ContainerDied","Data":"4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02"} Jul 23 21:58:13.707772 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.707683 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" event={"ID":"2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73","Type":"ContainerDied","Data":"297487d3d8043deda2b4d8e4a7aa18a624342ada9391bad8aee1745f95939acb"} Jul 23 21:58:13.707772 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.707702 2579 scope.go:117] "RemoveContainer" containerID="4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02" Jul 23 21:58:13.707916 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.707858 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm" Jul 23 21:58:13.735143 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.735118 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm"] Jul 23 21:58:13.741071 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.741020 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kvsrm"] Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782166 2579 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-cacerts\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782197 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-dns-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782215 2579 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-local-certs\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782231 2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-token\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782243 2579 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-kubeconfig\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782256 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwzvf\" (UniqueName: \"kubernetes.io/projected/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-kube-api-access-cwzvf\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:13.782273 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:13.782269 2579 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73-istio-csr-ca-configmap\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:14.228903 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:14.228724 2579 scope.go:117] "RemoveContainer" containerID="4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02" Jul 23 21:58:14.229240 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:14.229168 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02\": container with ID starting with 4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02 not found: ID does not exist" containerID="4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02" Jul 23 21:58:14.229240 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:14.229208 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02"} err="failed to get container status \"4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02\": rpc error: code = NotFound desc = could not find container \"4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02\": container with ID starting with 4bdaab37c6bb7f75de8e520b1842b2cb7d42e795a16768c1d102af9b35422a02 not found: ID does not exist" Jul 23 21:58:14.715854 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:14.715821 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" event={"ID":"342c3d0c-6c49-4af9-9237-d37885218558","Type":"ContainerStarted","Data":"0c2d30c6dda41ba8dc75068d6d13059fd2d0b1aa77bd8455333f6c28c8c33a1c"} Jul 23 21:58:14.716069 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:14.715924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:14.740864 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:14.740814 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" podStartSLOduration=2.5796070760000003 podStartE2EDuration="4.740786764s" totalImagesPullingTime="2.161179688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:14.738177335 +0000 UTC m=+407.497248366" watchObservedRunningTime="2026-07-23 21:58:14.740786764 +0000 UTC m=+407.499857781" Jul 23 21:58:15.679002 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:15.678967 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" path="/var/lib/kubelet/pods/2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73/volumes" Jul 23 21:58:15.722151 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:15.722117 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" event={"ID":"589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3","Type":"ContainerStarted","Data":"c6c6c58b6e8ae65026983f0d7b24bfe5b4aceef9cfbe73df6a747ceca3d74946"} Jul 23 21:58:15.722315 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:15.722189 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:15.749307 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:15.749263 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" podStartSLOduration=2.916435758 podStartE2EDuration="5.749249544s" totalImagesPullingTime="2.832813786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:15.747392721 +0000 UTC m=+408.506463733" watchObservedRunningTime="2026-07-23 21:58:15.749249544 +0000 UTC m=+408.508320555" Jul 23 21:58:16.660412 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:16.660383 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wft6m" Jul 23 21:58:16.663533 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:16.663511 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-z96zp" Jul 23 21:58:45.727691 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:45.727662 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qll9x" Jul 23 21:58:46.730794 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:46.730762 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-wkl4v" Jul 23 21:58:55.196423 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.196343 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chxcf"] Jul 23 21:58:55.196791 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.196775 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" containerName="discovery" Jul 23 21:58:55.196846 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.196793 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" containerName="discovery" Jul 23 21:58:55.196880 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.196870 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f4bfdc3-1e16-4b44-a1dc-7d12a5164d73" containerName="discovery" Jul 23 21:58:55.202890 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.202865 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:55.206183 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.206161 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b8bjl\"" Jul 23 21:58:55.206479 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.206446 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 21:58:55.206616 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.206585 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 21:58:55.210299 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.210265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chxcf"] Jul 23 21:58:55.257668 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.257643 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-278fx\" (UniqueName: \"kubernetes.io/projected/f0e696fa-ec3e-494c-9dda-933fdb4aafbd-kube-api-access-278fx\") pod \"kuadrant-operator-catalog-chxcf\" (UID: \"f0e696fa-ec3e-494c-9dda-933fdb4aafbd\") " pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:55.358640 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.358605 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-278fx\" (UniqueName: \"kubernetes.io/projected/f0e696fa-ec3e-494c-9dda-933fdb4aafbd-kube-api-access-278fx\") pod \"kuadrant-operator-catalog-chxcf\" (UID: \"f0e696fa-ec3e-494c-9dda-933fdb4aafbd\") " pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:55.373320 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.373286 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-278fx\" (UniqueName: \"kubernetes.io/projected/f0e696fa-ec3e-494c-9dda-933fdb4aafbd-kube-api-access-278fx\") pod \"kuadrant-operator-catalog-chxcf\" (UID: \"f0e696fa-ec3e-494c-9dda-933fdb4aafbd\") " pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:55.516344 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.516257 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:55.557913 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.557880 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chxcf"] Jul 23 21:58:55.652536 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.652508 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chxcf"] Jul 23 21:58:55.654691 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:55.654656 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e696fa_ec3e_494c_9dda_933fdb4aafbd.slice/crio-e54191bfaae1fcbff9fe6f4d591a2040dfe0fa2f8666b46d9cedbe7979f81f7f WatchSource:0}: Error finding container e54191bfaae1fcbff9fe6f4d591a2040dfe0fa2f8666b46d9cedbe7979f81f7f: Status 404 returned error can't find the container with id e54191bfaae1fcbff9fe6f4d591a2040dfe0fa2f8666b46d9cedbe7979f81f7f Jul 23 21:58:55.767902 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.767818 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nhchq"] Jul 23 21:58:55.770803 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.770786 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:58:55.780583 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.780559 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nhchq"] Jul 23 21:58:55.862724 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.862692 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5bsv\" (UniqueName: \"kubernetes.io/projected/ef78e830-5c2b-4c2a-80e5-1663e61a040e-kube-api-access-s5bsv\") pod \"kuadrant-operator-catalog-nhchq\" (UID: \"ef78e830-5c2b-4c2a-80e5-1663e61a040e\") " pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:58:55.918401 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.918366 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" event={"ID":"f0e696fa-ec3e-494c-9dda-933fdb4aafbd","Type":"ContainerStarted","Data":"e54191bfaae1fcbff9fe6f4d591a2040dfe0fa2f8666b46d9cedbe7979f81f7f"} Jul 23 21:58:55.963681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.963640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5bsv\" (UniqueName: \"kubernetes.io/projected/ef78e830-5c2b-4c2a-80e5-1663e61a040e-kube-api-access-s5bsv\") pod \"kuadrant-operator-catalog-nhchq\" (UID: \"ef78e830-5c2b-4c2a-80e5-1663e61a040e\") " pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:58:55.971793 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:55.971763 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5bsv\" (UniqueName: \"kubernetes.io/projected/ef78e830-5c2b-4c2a-80e5-1663e61a040e-kube-api-access-s5bsv\") pod \"kuadrant-operator-catalog-nhchq\" (UID: \"ef78e830-5c2b-4c2a-80e5-1663e61a040e\") " pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:58:56.081364 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:56.081279 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:58:56.225327 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:56.225265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nhchq"] Jul 23 21:58:56.242380 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:58:56.242341 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef78e830_5c2b_4c2a_80e5_1663e61a040e.slice/crio-9aec48cc37d55a403d59ad5da729897a7db4ef93ee681a2298e2be7b54d5704e WatchSource:0}: Error finding container 9aec48cc37d55a403d59ad5da729897a7db4ef93ee681a2298e2be7b54d5704e: Status 404 returned error can't find the container with id 9aec48cc37d55a403d59ad5da729897a7db4ef93ee681a2298e2be7b54d5704e Jul 23 21:58:56.927692 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:56.927653 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" event={"ID":"ef78e830-5c2b-4c2a-80e5-1663e61a040e","Type":"ContainerStarted","Data":"9aec48cc37d55a403d59ad5da729897a7db4ef93ee681a2298e2be7b54d5704e"} Jul 23 21:58:57.934941 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:57.934847 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" event={"ID":"f0e696fa-ec3e-494c-9dda-933fdb4aafbd","Type":"ContainerStarted","Data":"ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808"} Jul 23 21:58:57.934941 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:57.934910 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" podUID="f0e696fa-ec3e-494c-9dda-933fdb4aafbd" containerName="registry-server" containerID="cri-o://ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808" gracePeriod=2 Jul 23 21:58:57.936572 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:57.936543 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" event={"ID":"ef78e830-5c2b-4c2a-80e5-1663e61a040e","Type":"ContainerStarted","Data":"aa7011804ad1d912a7a0464eebd2c707bf67bcddbacd0ca37e717e6ea84965a8"} Jul 23 21:58:57.953674 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:57.953631 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" podStartSLOduration=0.943721009 podStartE2EDuration="2.953615735s" totalImagesPullingTime="2.009894726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:57.951677459 +0000 UTC m=+450.710748473" watchObservedRunningTime="2026-07-23 21:58:57.953615735 +0000 UTC m=+450.712686747" Jul 23 21:58:57.972392 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:57.972347 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" podStartSLOduration=1.547326549 podStartE2EDuration="2.972332574s" totalImagesPullingTime="1.425006025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:57.970985312 +0000 UTC m=+450.730056344" watchObservedRunningTime="2026-07-23 21:58:57.972332574 +0000 UTC m=+450.731403584" Jul 23 21:58:58.192301 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.192239 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:58.286790 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.286722 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-278fx\" (UniqueName: \"kubernetes.io/projected/f0e696fa-ec3e-494c-9dda-933fdb4aafbd-kube-api-access-278fx\") pod \"f0e696fa-ec3e-494c-9dda-933fdb4aafbd\" (UID: \"f0e696fa-ec3e-494c-9dda-933fdb4aafbd\") " Jul 23 21:58:58.288861 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.288840 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e696fa-ec3e-494c-9dda-933fdb4aafbd-kube-api-access-278fx" (OuterVolumeSpecName: "kube-api-access-278fx") pod "f0e696fa-ec3e-494c-9dda-933fdb4aafbd" (UID: "f0e696fa-ec3e-494c-9dda-933fdb4aafbd"). InnerVolumeSpecName "kube-api-access-278fx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:58:58.388129 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.388095 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-278fx\" (UniqueName: \"kubernetes.io/projected/f0e696fa-ec3e-494c-9dda-933fdb4aafbd-kube-api-access-278fx\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:58:58.943737 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.943641 2579 generic.go:356] "Generic (PLEG): container finished" podID="f0e696fa-ec3e-494c-9dda-933fdb4aafbd" containerID="ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808" exitCode=0 Jul 23 21:58:58.943737 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.943693 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" Jul 23 21:58:58.944247 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.943735 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" event={"ID":"f0e696fa-ec3e-494c-9dda-933fdb4aafbd","Type":"ContainerDied","Data":"ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808"} Jul 23 21:58:58.944247 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.943781 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-chxcf" event={"ID":"f0e696fa-ec3e-494c-9dda-933fdb4aafbd","Type":"ContainerDied","Data":"e54191bfaae1fcbff9fe6f4d591a2040dfe0fa2f8666b46d9cedbe7979f81f7f"} Jul 23 21:58:58.944247 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.943810 2579 scope.go:117] "RemoveContainer" containerID="ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808" Jul 23 21:58:58.953468 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.953451 2579 scope.go:117] "RemoveContainer" containerID="ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808" Jul 23 21:58:58.953713 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:58:58.953698 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808\": container with ID starting with ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808 not found: ID does not exist" containerID="ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808" Jul 23 21:58:58.953757 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.953720 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808"} err="failed to get container status \"ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808\": rpc error: code = NotFound desc = could not find container \"ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808\": container with ID starting with ccf605ea8788e8314452895d4ca01d6a13bd8321e13619ab0999c77480039808 not found: ID does not exist" Jul 23 21:58:58.966006 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.965981 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chxcf"] Jul 23 21:58:58.969123 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:58.969104 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-chxcf"] Jul 23 21:58:59.679230 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:58:59.679198 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e696fa-ec3e-494c-9dda-933fdb4aafbd" path="/var/lib/kubelet/pods/f0e696fa-ec3e-494c-9dda-933fdb4aafbd/volumes" Jul 23 21:59:06.082243 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:06.082176 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:59:06.082695 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:06.082281 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:59:06.106156 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:06.106128 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:59:07.007234 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:07.007209 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-nhchq" Jul 23 21:59:19.565362 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.565326 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b"] Jul 23 21:59:19.565840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.565737 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e696fa-ec3e-494c-9dda-933fdb4aafbd" containerName="registry-server" Jul 23 21:59:19.565840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.565770 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e696fa-ec3e-494c-9dda-933fdb4aafbd" containerName="registry-server" Jul 23 21:59:19.565840 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.565835 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e696fa-ec3e-494c-9dda-933fdb4aafbd" containerName="registry-server" Jul 23 21:59:19.570332 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.570316 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:19.573077 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.573046 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-b4nm6\"" Jul 23 21:59:19.583542 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.583520 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b"] Jul 23 21:59:19.682411 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.682383 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbnhr\" (UniqueName: \"kubernetes.io/projected/9c317d40-5abc-4c67-94c2-07977241132b-kube-api-access-jbnhr\") pod \"authorino-operator-75cc8d5cb7-2kz8b\" (UID: \"9c317d40-5abc-4c67-94c2-07977241132b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:19.783717 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.783684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbnhr\" (UniqueName: \"kubernetes.io/projected/9c317d40-5abc-4c67-94c2-07977241132b-kube-api-access-jbnhr\") pod \"authorino-operator-75cc8d5cb7-2kz8b\" (UID: \"9c317d40-5abc-4c67-94c2-07977241132b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:19.797533 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.797502 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbnhr\" (UniqueName: \"kubernetes.io/projected/9c317d40-5abc-4c67-94c2-07977241132b-kube-api-access-jbnhr\") pod \"authorino-operator-75cc8d5cb7-2kz8b\" (UID: \"9c317d40-5abc-4c67-94c2-07977241132b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:19.881178 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:19.881154 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:20.021193 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:20.021170 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b"] Jul 23 21:59:20.023597 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:20.023574 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c317d40_5abc_4c67_94c2_07977241132b.slice/crio-9792a9bd5307d012f5b5fe9876129781b0a8a27abc08f160d82109ea20606f61 WatchSource:0}: Error finding container 9792a9bd5307d012f5b5fe9876129781b0a8a27abc08f160d82109ea20606f61: Status 404 returned error can't find the container with id 9792a9bd5307d012f5b5fe9876129781b0a8a27abc08f160d82109ea20606f61 Jul 23 21:59:20.052203 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:20.052172 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" event={"ID":"9c317d40-5abc-4c67-94c2-07977241132b","Type":"ContainerStarted","Data":"9792a9bd5307d012f5b5fe9876129781b0a8a27abc08f160d82109ea20606f61"} Jul 23 21:59:23.077191 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.077145 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" event={"ID":"9c317d40-5abc-4c67-94c2-07977241132b","Type":"ContainerStarted","Data":"3ddd3df8dce0822ac196a2f4929a31da792e42fd0d0d72d399aefe354e963865"} Jul 23 21:59:23.077589 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.077234 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:23.095589 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.095545 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" podStartSLOduration=1.784288669 podStartE2EDuration="4.095531343s" totalImagesPullingTime="2.311242674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:23.094320927 +0000 UTC m=+475.853391959" watchObservedRunningTime="2026-07-23 21:59:23.095531343 +0000 UTC m=+475.854602354" Jul 23 21:59:23.862838 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.862808 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc"] Jul 23 21:59:23.866354 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.866337 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:23.869778 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.869756 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-q2prr\"" Jul 23 21:59:23.869890 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.869851 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 23 21:59:23.882494 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:23.882466 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc"] Jul 23 21:59:24.024923 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:24.024883 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s55wp\" (UniqueName: \"kubernetes.io/projected/cefbb45b-94ee-4d83-8e39-8b01876d871c-kube-api-access-s55wp\") pod \"dns-operator-controller-manager-864dd78c4c-gksqc\" (UID: \"cefbb45b-94ee-4d83-8e39-8b01876d871c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:24.126253 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:24.126177 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s55wp\" (UniqueName: \"kubernetes.io/projected/cefbb45b-94ee-4d83-8e39-8b01876d871c-kube-api-access-s55wp\") pod \"dns-operator-controller-manager-864dd78c4c-gksqc\" (UID: \"cefbb45b-94ee-4d83-8e39-8b01876d871c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:24.136603 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:24.136580 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s55wp\" (UniqueName: \"kubernetes.io/projected/cefbb45b-94ee-4d83-8e39-8b01876d871c-kube-api-access-s55wp\") pod \"dns-operator-controller-manager-864dd78c4c-gksqc\" (UID: \"cefbb45b-94ee-4d83-8e39-8b01876d871c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:24.176384 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:24.176357 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:24.316490 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:24.316467 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc"] Jul 23 21:59:24.318460 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:24.318431 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcefbb45b_94ee_4d83_8e39_8b01876d871c.slice/crio-54bbc052c7e0d88ca2bce9c062f53efc81a1822e65c83d2ccbc7644c1d289c71 WatchSource:0}: Error finding container 54bbc052c7e0d88ca2bce9c062f53efc81a1822e65c83d2ccbc7644c1d289c71: Status 404 returned error can't find the container with id 54bbc052c7e0d88ca2bce9c062f53efc81a1822e65c83d2ccbc7644c1d289c71 Jul 23 21:59:25.089598 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:25.089559 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" event={"ID":"cefbb45b-94ee-4d83-8e39-8b01876d871c","Type":"ContainerStarted","Data":"54bbc052c7e0d88ca2bce9c062f53efc81a1822e65c83d2ccbc7644c1d289c71"} Jul 23 21:59:26.096561 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:26.096512 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" event={"ID":"cefbb45b-94ee-4d83-8e39-8b01876d871c","Type":"ContainerStarted","Data":"32482975066a9a75ef8688090dac69acdd405b639c0ac24aa842722a4acabe55"} Jul 23 21:59:26.096919 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:26.096576 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:26.124684 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:26.124589 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" podStartSLOduration=1.585518171 podStartE2EDuration="3.124572664s" totalImagesPullingTime="1.539054493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:26.122640759 +0000 UTC m=+478.881711771" watchObservedRunningTime="2026-07-23 21:59:26.124572664 +0000 UTC m=+478.883643675" Jul 23 21:59:28.834966 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.834933 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd"] Jul 23 21:59:28.843131 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.843103 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:28.846037 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.846008 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hkd2b\"" Jul 23 21:59:28.855990 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.855965 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd"] Jul 23 21:59:28.867588 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.867560 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:28.867768 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.867745 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84dzg\" (UniqueName: \"kubernetes.io/projected/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-kube-api-access-84dzg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:28.968993 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.968959 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84dzg\" (UniqueName: \"kubernetes.io/projected/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-kube-api-access-84dzg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:28.969183 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.969022 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:28.969390 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.969371 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:28.978673 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:28.978651 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84dzg\" (UniqueName: \"kubernetes.io/projected/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-kube-api-access-84dzg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:29.153583 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:29.153553 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:29.303184 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:29.303158 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd"] Jul 23 21:59:29.305251 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:29.305225 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01597b9e_46c8_48f3_9b9d_fa3d1de3ab9e.slice/crio-d04196bb043ad9a1d275e203a3d2f5f6210fbcea7e4b50a2f3c72343715e04fe WatchSource:0}: Error finding container d04196bb043ad9a1d275e203a3d2f5f6210fbcea7e4b50a2f3c72343715e04fe: Status 404 returned error can't find the container with id d04196bb043ad9a1d275e203a3d2f5f6210fbcea7e4b50a2f3c72343715e04fe Jul 23 21:59:30.121684 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.121644 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" event={"ID":"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e","Type":"ContainerStarted","Data":"d04196bb043ad9a1d275e203a3d2f5f6210fbcea7e4b50a2f3c72343715e04fe"} Jul 23 21:59:30.729259 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.729223 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8"] Jul 23 21:59:30.733532 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.733504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:30.737509 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.737483 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-74z9d\"" Jul 23 21:59:30.750885 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.750858 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8"] Jul 23 21:59:30.789339 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.789241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2tkx\" (UniqueName: \"kubernetes.io/projected/dab49ca5-8743-4374-a09a-053406aca37b-kube-api-access-g2tkx\") pod \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" (UID: \"dab49ca5-8743-4374-a09a-053406aca37b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:30.889901 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.889863 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2tkx\" (UniqueName: \"kubernetes.io/projected/dab49ca5-8743-4374-a09a-053406aca37b-kube-api-access-g2tkx\") pod \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" (UID: \"dab49ca5-8743-4374-a09a-053406aca37b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:30.907001 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:30.906898 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2tkx\" (UniqueName: \"kubernetes.io/projected/dab49ca5-8743-4374-a09a-053406aca37b-kube-api-access-g2tkx\") pod \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" (UID: \"dab49ca5-8743-4374-a09a-053406aca37b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:31.049591 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:31.049511 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:31.205654 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:31.205609 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8"] Jul 23 21:59:31.210459 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:31.210416 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab49ca5_8743_4374_a09a_053406aca37b.slice/crio-babc89f517dc2c1946aff291a4480487e57ea153ee36caf6cd7cf65d45043e07 WatchSource:0}: Error finding container babc89f517dc2c1946aff291a4480487e57ea153ee36caf6cd7cf65d45043e07: Status 404 returned error can't find the container with id babc89f517dc2c1946aff291a4480487e57ea153ee36caf6cd7cf65d45043e07 Jul 23 21:59:32.136159 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:32.136119 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" event={"ID":"dab49ca5-8743-4374-a09a-053406aca37b","Type":"ContainerStarted","Data":"babc89f517dc2c1946aff291a4480487e57ea153ee36caf6cd7cf65d45043e07"} Jul 23 21:59:34.084724 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.084633 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-2kz8b" Jul 23 21:59:34.150650 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.150596 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" event={"ID":"dab49ca5-8743-4374-a09a-053406aca37b","Type":"ContainerStarted","Data":"60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd"} Jul 23 21:59:34.150841 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.150782 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:34.152360 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.152332 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" event={"ID":"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e","Type":"ContainerStarted","Data":"bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4"} Jul 23 21:59:34.152503 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.152473 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:34.172482 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.172434 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" podStartSLOduration=1.551132583 podStartE2EDuration="4.172418771s" totalImagesPullingTime="2.621286188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:34.169851216 +0000 UTC m=+486.928922246" watchObservedRunningTime="2026-07-23 21:59:34.172418771 +0000 UTC m=+486.931489781" Jul 23 21:59:34.198728 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:34.198677 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" podStartSLOduration=1.6586850960000001 podStartE2EDuration="6.198663417s" totalImagesPullingTime="4.539978321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:34.196877583 +0000 UTC m=+486.955948613" watchObservedRunningTime="2026-07-23 21:59:34.198663417 +0000 UTC m=+486.957734429" Jul 23 21:59:36.610216 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.610183 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw"] Jul 23 21:59:36.614292 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.614265 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.617011 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.616990 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 23 21:59:36.617222 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.617198 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zk25n\"" Jul 23 21:59:36.617318 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.617222 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 23 21:59:36.624682 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.624659 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw"] Jul 23 21:59:36.635963 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.635940 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7a87b00-f781-4f74-85ae-2c94bc2165cd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.636152 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.635985 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a87b00-f781-4f74-85ae-2c94bc2165cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.636152 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.636099 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9rv\" (UniqueName: \"kubernetes.io/projected/e7a87b00-f781-4f74-85ae-2c94bc2165cd-kube-api-access-jp9rv\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.737336 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.737304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a87b00-f781-4f74-85ae-2c94bc2165cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.737491 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.737359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9rv\" (UniqueName: \"kubernetes.io/projected/e7a87b00-f781-4f74-85ae-2c94bc2165cd-kube-api-access-jp9rv\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.737491 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:36.737441 2579 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 23 21:59:36.737637 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:36.737501 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7a87b00-f781-4f74-85ae-2c94bc2165cd-plugin-serving-cert podName:e7a87b00-f781-4f74-85ae-2c94bc2165cd nodeName:}" failed. No retries permitted until 2026-07-23 21:59:37.237480744 +0000 UTC m=+489.996551742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e7a87b00-f781-4f74-85ae-2c94bc2165cd-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-nh6zw" (UID: "e7a87b00-f781-4f74-85ae-2c94bc2165cd") : secret "plugin-serving-cert" not found Jul 23 21:59:36.737637 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.737540 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7a87b00-f781-4f74-85ae-2c94bc2165cd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.738228 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.738211 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e7a87b00-f781-4f74-85ae-2c94bc2165cd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:36.746127 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:36.746105 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9rv\" (UniqueName: \"kubernetes.io/projected/e7a87b00-f781-4f74-85ae-2c94bc2165cd-kube-api-access-jp9rv\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:37.105076 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.105026 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gksqc" Jul 23 21:59:37.241245 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.241212 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a87b00-f781-4f74-85ae-2c94bc2165cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:37.243523 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.243503 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e7a87b00-f781-4f74-85ae-2c94bc2165cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-nh6zw\" (UID: \"e7a87b00-f781-4f74-85ae-2c94bc2165cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:37.341563 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.341516 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8595d64bf8-27gtq"] Jul 23 21:59:37.345579 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.345555 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.358210 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.358150 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8595d64bf8-27gtq"] Jul 23 21:59:37.443295 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443262 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-oauth-serving-cert\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.443448 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-service-ca\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.443448 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443339 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-serving-cert\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.443448 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-trusted-ca-bundle\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.443448 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcp7x\" (UniqueName: \"kubernetes.io/projected/e86782f7-017e-4131-b7d1-b7fb1ca4a209-kube-api-access-vcp7x\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.443594 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443444 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-oauth-config\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.443594 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.443518 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-config\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.523922 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.523892 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" Jul 23 21:59:37.544409 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544382 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-trusted-ca-bundle\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.544528 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544415 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcp7x\" (UniqueName: \"kubernetes.io/projected/e86782f7-017e-4131-b7d1-b7fb1ca4a209-kube-api-access-vcp7x\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.544528 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544434 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-oauth-config\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.544528 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544478 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-config\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.544528 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-oauth-serving-cert\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.544736 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544563 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-service-ca\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.544736 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.544606 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-serving-cert\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.545378 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.545326 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-config\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.545619 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.545594 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-oauth-serving-cert\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.545775 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.545665 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-trusted-ca-bundle\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.546136 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.546113 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e86782f7-017e-4131-b7d1-b7fb1ca4a209-service-ca\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.547295 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.547260 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-serving-cert\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.547653 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.547632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e86782f7-017e-4131-b7d1-b7fb1ca4a209-console-oauth-config\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.553508 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.553487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcp7x\" (UniqueName: \"kubernetes.io/projected/e86782f7-017e-4131-b7d1-b7fb1ca4a209-kube-api-access-vcp7x\") pod \"console-8595d64bf8-27gtq\" (UID: \"e86782f7-017e-4131-b7d1-b7fb1ca4a209\") " pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.657321 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.657293 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:37.684837 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.684804 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw"] Jul 23 21:59:37.686225 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:37.686201 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7a87b00_f781_4f74_85ae_2c94bc2165cd.slice/crio-c81105ccc1633d57cf8c94d7bfd04845156a99eb3ce8212547092ab9bb1ffeec WatchSource:0}: Error finding container c81105ccc1633d57cf8c94d7bfd04845156a99eb3ce8212547092ab9bb1ffeec: Status 404 returned error can't find the container with id c81105ccc1633d57cf8c94d7bfd04845156a99eb3ce8212547092ab9bb1ffeec Jul 23 21:59:37.812620 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:37.812572 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8595d64bf8-27gtq"] Jul 23 21:59:37.815492 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:37.815465 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86782f7_017e_4131_b7d1_b7fb1ca4a209.slice/crio-cff469b1c6f5d9b62ad94311f58f27658bab860f880864ee6fced3a98eb66e45 WatchSource:0}: Error finding container cff469b1c6f5d9b62ad94311f58f27658bab860f880864ee6fced3a98eb66e45: Status 404 returned error can't find the container with id cff469b1c6f5d9b62ad94311f58f27658bab860f880864ee6fced3a98eb66e45 Jul 23 21:59:38.174087 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:38.173983 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8595d64bf8-27gtq" event={"ID":"e86782f7-017e-4131-b7d1-b7fb1ca4a209","Type":"ContainerStarted","Data":"d383f884effa6d016109d8593200c34dd67891d87a0cacb100ce299e5387af81"} Jul 23 21:59:38.174087 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:38.174031 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8595d64bf8-27gtq" event={"ID":"e86782f7-017e-4131-b7d1-b7fb1ca4a209","Type":"ContainerStarted","Data":"cff469b1c6f5d9b62ad94311f58f27658bab860f880864ee6fced3a98eb66e45"} Jul 23 21:59:38.175267 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:38.175242 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" event={"ID":"e7a87b00-f781-4f74-85ae-2c94bc2165cd","Type":"ContainerStarted","Data":"c81105ccc1633d57cf8c94d7bfd04845156a99eb3ce8212547092ab9bb1ffeec"} Jul 23 21:59:38.196873 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:38.196819 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8595d64bf8-27gtq" podStartSLOduration=1.196801092 podStartE2EDuration="1.196801092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:38.194878652 +0000 UTC m=+490.953949691" watchObservedRunningTime="2026-07-23 21:59:38.196801092 +0000 UTC m=+490.955872081" Jul 23 21:59:45.160406 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:45.160373 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:45.160858 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:45.160454 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:46.140913 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.140881 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l"] Jul 23 21:59:46.151600 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.151574 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.156220 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.156194 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l"] Jul 23 21:59:46.225695 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.225663 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpbhr\" (UniqueName: \"kubernetes.io/projected/f127d205-8686-4a94-af20-084803277ee0-kube-api-access-bpbhr\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.226131 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.225712 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f127d205-8686-4a94-af20-084803277ee0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.327178 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.327142 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpbhr\" (UniqueName: \"kubernetes.io/projected/f127d205-8686-4a94-af20-084803277ee0-kube-api-access-bpbhr\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.327178 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.327185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f127d205-8686-4a94-af20-084803277ee0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.327640 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.327618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f127d205-8686-4a94-af20-084803277ee0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.348340 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.348283 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpbhr\" (UniqueName: \"kubernetes.io/projected/f127d205-8686-4a94-af20-084803277ee0-kube-api-access-bpbhr\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:46.466509 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:46.466416 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 21:59:47.065841 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.065807 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l"] Jul 23 21:59:47.069993 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.069965 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l"] Jul 23 21:59:47.081452 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.081420 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd"] Jul 23 21:59:47.081723 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.081649 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" containerName="manager" containerID="cri-o://bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4" gracePeriod=2 Jul 23 21:59:47.084850 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.084576 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:47.086252 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.086225 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd"] Jul 23 21:59:47.137044 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.136986 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8"] Jul 23 21:59:47.137613 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.137293 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd"] Jul 23 21:59:47.138499 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.138469 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" containerName="manager" Jul 23 21:59:47.138499 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.138499 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" containerName="manager" Jul 23 21:59:47.138788 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.138761 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" containerName="manager" Jul 23 21:59:47.138869 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.138842 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" podUID="dab49ca5-8743-4374-a09a-053406aca37b" containerName="manager" containerID="cri-o://60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd" gracePeriod=2 Jul 23 21:59:47.142916 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.142879 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:47.147599 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.147200 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.150525 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.150191 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8"] Jul 23 21:59:47.154976 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.154951 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd"] Jul 23 21:59:47.155243 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.155217 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:47.158291 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.158259 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:47.164776 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.164752 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z"] Jul 23 21:59:47.165385 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.165367 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dab49ca5-8743-4374-a09a-053406aca37b" containerName="manager" Jul 23 21:59:47.165527 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.165487 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="dab49ca5-8743-4374-a09a-053406aca37b" containerName="manager" Jul 23 21:59:47.165681 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.165667 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="dab49ca5-8743-4374-a09a-053406aca37b" containerName="manager" Jul 23 21:59:47.169351 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.169331 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 21:59:47.183650 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.183607 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z"] Jul 23 21:59:47.191964 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.191882 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:47.225001 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.224958 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:47.236518 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.236485 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vgpt\" (UniqueName: \"kubernetes.io/projected/306be4f4-34a1-48cf-bfcd-d512b29fb078-kube-api-access-7vgpt\") pod \"limitador-operator-controller-manager-859d7b78bd-svb4z\" (UID: \"306be4f4-34a1-48cf-bfcd-d512b29fb078\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 21:59:47.236905 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.236566 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rbpt\" (UniqueName: \"kubernetes.io/projected/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-kube-api-access-7rbpt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbmsd\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.236905 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.236645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbmsd\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.337467 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.337387 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rbpt\" (UniqueName: \"kubernetes.io/projected/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-kube-api-access-7rbpt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbmsd\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.337467 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.337460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbmsd\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.337677 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.337525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vgpt\" (UniqueName: \"kubernetes.io/projected/306be4f4-34a1-48cf-bfcd-d512b29fb078-kube-api-access-7vgpt\") pod \"limitador-operator-controller-manager-859d7b78bd-svb4z\" (UID: \"306be4f4-34a1-48cf-bfcd-d512b29fb078\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 21:59:47.337860 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.337835 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbmsd\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.347689 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.347659 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rbpt\" (UniqueName: \"kubernetes.io/projected/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-kube-api-access-7rbpt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fbmsd\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.347823 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.347705 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vgpt\" (UniqueName: \"kubernetes.io/projected/306be4f4-34a1-48cf-bfcd-d512b29fb078-kube-api-access-7vgpt\") pod \"limitador-operator-controller-manager-859d7b78bd-svb4z\" (UID: \"306be4f4-34a1-48cf-bfcd-d512b29fb078\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 21:59:47.504246 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.504190 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:47.513631 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.513610 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 21:59:47.658470 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.658431 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:47.658670 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.658499 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:47.664152 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:47.664129 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:47.666654 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.666620 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:47.669150 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:47.669122 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:48.254425 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:48.254392 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8595d64bf8-27gtq" Jul 23 21:59:48.356110 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:48.356083 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bd8698999-qqh5p"] Jul 23 21:59:59.150389 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.150351 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.153223 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.153200 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:59.153349 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.153201 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.155859 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.155826 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:59.160557 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.160514 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.165153 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.165130 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z"] Jul 23 21:59:59.167622 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:59.167594 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306be4f4_34a1_48cf_bfcd_d512b29fb078.slice/crio-3c8ad87c8eb6e030ae1a760b1228699b1859858c3731ca0fb1acb35388155391 WatchSource:0}: Error finding container 3c8ad87c8eb6e030ae1a760b1228699b1859858c3731ca0fb1acb35388155391: Status 404 returned error can't find the container with id 3c8ad87c8eb6e030ae1a760b1228699b1859858c3731ca0fb1acb35388155391 Jul 23 21:59:59.200488 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.200402 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd"] Jul 23 21:59:59.205641 ip-10-0-139-32 kubenswrapper[2579]: W0723 21:59:59.205615 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5450fcfa_a7c6_4fc7_8df7_6e96fe5c4b71.slice/crio-2bd3c4156f8c0f3c19ab9861657cd579dcb3e0f92c262dacf566be64445ff16c WatchSource:0}: Error finding container 2bd3c4156f8c0f3c19ab9861657cd579dcb3e0f92c262dacf566be64445ff16c: Status 404 returned error can't find the container with id 2bd3c4156f8c0f3c19ab9861657cd579dcb3e0f92c262dacf566be64445ff16c Jul 23 21:59:59.259033 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.259005 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-extensions-socket-volume\") pod \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " Jul 23 21:59:59.259169 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.259100 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84dzg\" (UniqueName: \"kubernetes.io/projected/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-kube-api-access-84dzg\") pod \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\" (UID: \"01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e\") " Jul 23 21:59:59.259169 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.259141 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2tkx\" (UniqueName: \"kubernetes.io/projected/dab49ca5-8743-4374-a09a-053406aca37b-kube-api-access-g2tkx\") pod \"dab49ca5-8743-4374-a09a-053406aca37b\" (UID: \"dab49ca5-8743-4374-a09a-053406aca37b\") " Jul 23 21:59:59.259541 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.259518 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" (UID: "01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:59.259619 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.259604 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-extensions-socket-volume\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:59:59.261366 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.261344 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-kube-api-access-84dzg" (OuterVolumeSpecName: "kube-api-access-84dzg") pod "01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" (UID: "01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e"). InnerVolumeSpecName "kube-api-access-84dzg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:59:59.262334 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.262306 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dab49ca5-8743-4374-a09a-053406aca37b-kube-api-access-g2tkx" (OuterVolumeSpecName: "kube-api-access-g2tkx") pod "dab49ca5-8743-4374-a09a-053406aca37b" (UID: "dab49ca5-8743-4374-a09a-053406aca37b"). InnerVolumeSpecName "kube-api-access-g2tkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:59:59.318688 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.318656 2579 generic.go:356] "Generic (PLEG): container finished" podID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" containerID="bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4" exitCode=0 Jul 23 21:59:59.318866 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.318695 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.318866 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.318758 2579 scope.go:117] "RemoveContainer" containerID="bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4" Jul 23 21:59:59.320533 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.320506 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" event={"ID":"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71","Type":"ContainerStarted","Data":"57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2"} Jul 23 21:59:59.320882 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.320557 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" event={"ID":"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71","Type":"ContainerStarted","Data":"2bd3c4156f8c0f3c19ab9861657cd579dcb3e0f92c262dacf566be64445ff16c"} Jul 23 21:59:59.320882 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.320588 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 21:59:59.322450 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.322425 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" event={"ID":"306be4f4-34a1-48cf-bfcd-d512b29fb078","Type":"ContainerStarted","Data":"97a3d6e96351b2dd6515aad6ad849d4c629a8825a6f175b18ae84f1df7355042"} Jul 23 21:59:59.322593 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.322474 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" event={"ID":"306be4f4-34a1-48cf-bfcd-d512b29fb078","Type":"ContainerStarted","Data":"3c8ad87c8eb6e030ae1a760b1228699b1859858c3731ca0fb1acb35388155391"} Jul 23 21:59:59.322593 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.322500 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 21:59:59.324126 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.324104 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" event={"ID":"e7a87b00-f781-4f74-85ae-2c94bc2165cd","Type":"ContainerStarted","Data":"5454bda78f6d42c634ad166b4c1f1158d013ab78c0dcd501e3970e446baffac2"} Jul 23 21:59:59.325270 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.325240 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:59.325603 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.325585 2579 generic.go:356] "Generic (PLEG): container finished" podID="dab49ca5-8743-4374-a09a-053406aca37b" containerID="60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd" exitCode=0 Jul 23 21:59:59.325677 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.325612 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:59.328918 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.328901 2579 scope.go:117] "RemoveContainer" containerID="bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4" Jul 23 21:59:59.329190 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.329172 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4\": container with ID starting with bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4 not found: ID does not exist" containerID="bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4" Jul 23 21:59:59.329266 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.329198 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4"} err="failed to get container status \"bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4\": rpc error: code = NotFound desc = could not find container \"bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4\": container with ID starting with bf7c8d137be68e6b7bc7333a8483063e150500be2e73af902d58b7df6e9265c4 not found: ID does not exist" Jul 23 21:59:59.329266 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.329213 2579 scope.go:117] "RemoveContainer" containerID="60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd" Jul 23 21:59:59.330135 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.330032 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.333607 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.333581 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.338143 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.338047 2579 scope.go:117] "RemoveContainer" containerID="60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd" Jul 23 21:59:59.338366 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.338349 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd\": container with ID starting with 60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd not found: ID does not exist" containerID="60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd" Jul 23 21:59:59.338438 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.338376 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd"} err="failed to get container status \"60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd\": rpc error: code = NotFound desc = could not find container \"60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd\": container with ID starting with 60e46f2eef9b480192085c0d48d67ea705d76a37cdfae6a17f8be157fe9ea3fd not found: ID does not exist" Jul 23 21:59:59.360160 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.360133 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84dzg\" (UniqueName: \"kubernetes.io/projected/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e-kube-api-access-84dzg\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:59:59.360160 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.360154 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2tkx\" (UniqueName: \"kubernetes.io/projected/dab49ca5-8743-4374-a09a-053406aca37b-kube-api-access-g2tkx\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 21:59:59.365388 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.365340 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" podStartSLOduration=12.365324321 podStartE2EDuration="12.365324321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:59.364152399 +0000 UTC m=+512.123223410" watchObservedRunningTime="2026-07-23 21:59:59.365324321 +0000 UTC m=+512.124395333" Jul 23 21:59:59.387305 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.387264 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-nh6zw" podStartSLOduration=2.050256501 podStartE2EDuration="23.387250288s" totalImagesPullingTime="21.336993787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:59.3869958 +0000 UTC m=+512.146066812" watchObservedRunningTime="2026-07-23 21:59:59.387250288 +0000 UTC m=+512.146321298" Jul 23 21:59:59.429582 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.429522 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" podStartSLOduration=12.429504617 podStartE2EDuration="12.429504617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:59:59.425603109 +0000 UTC m=+512.184674139" watchObservedRunningTime="2026-07-23 21:59:59.429504617 +0000 UTC m=+512.188575629" Jul 23 21:59:59.431170 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.431135 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:59.441258 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.441220 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rfzgd\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rfzgd" Jul 23 21:59:59.445281 ip-10-0-139-32 kubenswrapper[2579]: E0723 21:59:59.445253 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-jfbv8\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="dab49ca5-8743-4374-a09a-053406aca37b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jfbv8" Jul 23 21:59:59.679736 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.679697 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e" path="/var/lib/kubelet/pods/01597b9e-46c8-48f3-9b9d-fa3d1de3ab9e/volumes" Jul 23 21:59:59.680135 ip-10-0-139-32 kubenswrapper[2579]: I0723 21:59:59.680116 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dab49ca5-8743-4374-a09a-053406aca37b" path="/var/lib/kubelet/pods/dab49ca5-8743-4374-a09a-053406aca37b/volumes" Jul 23 22:00:01.522416 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:01.522371 2579 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-lgq8l_kuadrant-system_f127d205-8686-4a94-af20-084803277ee0_0(540fcaa45e6f9837041498cf825e5ce10ed0df69cbb11b664139befcb98c7f6a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-lgq8l to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 23 22:00:01.522729 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:01.522439 2579 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-lgq8l_kuadrant-system_f127d205-8686-4a94-af20-084803277ee0_0(540fcaa45e6f9837041498cf825e5ce10ed0df69cbb11b664139befcb98c7f6a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-lgq8l to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:02.343845 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.343816 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:02.346189 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:02.346165 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="f127d205-8686-4a94-af20-084803277ee0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:02.351875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.351853 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:02.354111 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:02.354082 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="f127d205-8686-4a94-af20-084803277ee0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:02.492103 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.492071 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpbhr\" (UniqueName: \"kubernetes.io/projected/f127d205-8686-4a94-af20-084803277ee0-kube-api-access-bpbhr\") pod \"f127d205-8686-4a94-af20-084803277ee0\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " Jul 23 22:00:02.492300 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.492133 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f127d205-8686-4a94-af20-084803277ee0-extensions-socket-volume\") pod \"f127d205-8686-4a94-af20-084803277ee0\" (UID: \"f127d205-8686-4a94-af20-084803277ee0\") " Jul 23 22:00:02.492372 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.492353 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f127d205-8686-4a94-af20-084803277ee0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f127d205-8686-4a94-af20-084803277ee0" (UID: "f127d205-8686-4a94-af20-084803277ee0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:00:02.494219 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.494190 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f127d205-8686-4a94-af20-084803277ee0-kube-api-access-bpbhr" (OuterVolumeSpecName: "kube-api-access-bpbhr") pod "f127d205-8686-4a94-af20-084803277ee0" (UID: "f127d205-8686-4a94-af20-084803277ee0"). InnerVolumeSpecName "kube-api-access-bpbhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:00:02.592728 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.592695 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpbhr\" (UniqueName: \"kubernetes.io/projected/f127d205-8686-4a94-af20-084803277ee0-kube-api-access-bpbhr\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:02.592728 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:02.592724 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f127d205-8686-4a94-af20-084803277ee0-extensions-socket-volume\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:03.349522 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:03.349490 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:03.352084 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:03.352043 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="f127d205-8686-4a94-af20-084803277ee0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:03.357428 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:03.357401 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-lgq8l\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="f127d205-8686-4a94-af20-084803277ee0" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lgq8l" Jul 23 22:00:03.679178 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:03.679140 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f127d205-8686-4a94-af20-084803277ee0" path="/var/lib/kubelet/pods/f127d205-8686-4a94-af20-084803277ee0/volumes" Jul 23 22:00:04.012746 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.012655 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd"] Jul 23 22:00:04.012928 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.012876 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" podUID="5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" containerName="manager" containerID="cri-o://57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2" gracePeriod=10 Jul 23 22:00:04.014626 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.014600 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 22:00:04.271443 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.271375 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 22:00:04.292404 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.292377 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458"] Jul 23 22:00:04.292819 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.292804 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" containerName="manager" Jul 23 22:00:04.292819 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.292820 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" containerName="manager" Jul 23 22:00:04.292909 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.292880 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" containerName="manager" Jul 23 22:00:04.295141 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.295121 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.310533 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.310477 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458"] Jul 23 22:00:04.357010 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.356979 2579 generic.go:356] "Generic (PLEG): container finished" podID="5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" containerID="57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2" exitCode=0 Jul 23 22:00:04.357177 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.357027 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" Jul 23 22:00:04.357177 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.357078 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" event={"ID":"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71","Type":"ContainerDied","Data":"57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2"} Jul 23 22:00:04.357177 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.357129 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd" event={"ID":"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71","Type":"ContainerDied","Data":"2bd3c4156f8c0f3c19ab9861657cd579dcb3e0f92c262dacf566be64445ff16c"} Jul 23 22:00:04.357177 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.357152 2579 scope.go:117] "RemoveContainer" containerID="57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2" Jul 23 22:00:04.366274 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.366258 2579 scope.go:117] "RemoveContainer" containerID="57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2" Jul 23 22:00:04.366513 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:04.366494 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2\": container with ID starting with 57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2 not found: ID does not exist" containerID="57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2" Jul 23 22:00:04.366575 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.366526 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2"} err="failed to get container status \"57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2\": rpc error: code = NotFound desc = could not find container \"57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2\": container with ID starting with 57210cbb8894dc060324599d2ca87aa290017facc520a473bb40ca14834984a2 not found: ID does not exist" Jul 23 22:00:04.408906 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.408872 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-extensions-socket-volume\") pod \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " Jul 23 22:00:04.409088 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.408923 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rbpt\" (UniqueName: \"kubernetes.io/projected/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-kube-api-access-7rbpt\") pod \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\" (UID: \"5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71\") " Jul 23 22:00:04.409163 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.409139 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08720b79-3f36-4808-a4c4-d8e580ba6a06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5w458\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.409309 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.409281 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" (UID: "5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:00:04.409374 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.409304 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmvvw\" (UniqueName: \"kubernetes.io/projected/08720b79-3f36-4808-a4c4-d8e580ba6a06-kube-api-access-kmvvw\") pod \"kuadrant-operator-controller-manager-86878c769f-5w458\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.409412 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.409381 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-extensions-socket-volume\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:04.410994 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.410964 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-kube-api-access-7rbpt" (OuterVolumeSpecName: "kube-api-access-7rbpt") pod "5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" (UID: "5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71"). InnerVolumeSpecName "kube-api-access-7rbpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:00:04.510705 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.510667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08720b79-3f36-4808-a4c4-d8e580ba6a06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5w458\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.510900 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.510806 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmvvw\" (UniqueName: \"kubernetes.io/projected/08720b79-3f36-4808-a4c4-d8e580ba6a06-kube-api-access-kmvvw\") pod \"kuadrant-operator-controller-manager-86878c769f-5w458\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.510900 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.510856 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rbpt\" (UniqueName: \"kubernetes.io/projected/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71-kube-api-access-7rbpt\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:04.511116 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.511045 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08720b79-3f36-4808-a4c4-d8e580ba6a06-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5w458\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.519225 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.519199 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmvvw\" (UniqueName: \"kubernetes.io/projected/08720b79-3f36-4808-a4c4-d8e580ba6a06-kube-api-access-kmvvw\") pod \"kuadrant-operator-controller-manager-86878c769f-5w458\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.608333 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.608304 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:04.689092 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.688982 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd"] Jul 23 22:00:04.692170 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.692148 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fbmsd"] Jul 23 22:00:04.751312 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:04.751288 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458"] Jul 23 22:00:04.753226 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:00:04.753201 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08720b79_3f36_4808_a4c4_d8e580ba6a06.slice/crio-e6ef20dccadc2569bf4ad0912ed2cb6c0497cc3372286db6116ace095845c9fb WatchSource:0}: Error finding container e6ef20dccadc2569bf4ad0912ed2cb6c0497cc3372286db6116ace095845c9fb: Status 404 returned error can't find the container with id e6ef20dccadc2569bf4ad0912ed2cb6c0497cc3372286db6116ace095845c9fb Jul 23 22:00:05.367564 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:05.367523 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" event={"ID":"08720b79-3f36-4808-a4c4-d8e580ba6a06","Type":"ContainerStarted","Data":"31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768"} Jul 23 22:00:05.367746 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:05.367645 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" event={"ID":"08720b79-3f36-4808-a4c4-d8e580ba6a06","Type":"ContainerStarted","Data":"e6ef20dccadc2569bf4ad0912ed2cb6c0497cc3372286db6116ace095845c9fb"} Jul 23 22:00:05.367746 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:05.367682 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:05.408810 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:05.408750 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" podStartSLOduration=1.408732541 podStartE2EDuration="1.408732541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:05.405366128 +0000 UTC m=+518.164437158" watchObservedRunningTime="2026-07-23 22:00:05.408732541 +0000 UTC m=+518.167803552" Jul 23 22:00:05.680020 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:05.679942 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71" path="/var/lib/kubelet/pods/5450fcfa-a7c6-4fc7-8df7-6e96fe5c4b71/volumes" Jul 23 22:00:10.336735 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:10.336702 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-svb4z" Jul 23 22:00:13.381387 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.381328 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bd8698999-qqh5p" podUID="6a759735-72a3-4679-b9d1-794aa70bad6c" containerName="console" containerID="cri-o://747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b" gracePeriod=15 Jul 23 22:00:13.633652 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.633581 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 22:00:13.693842 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.693813 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-trusted-ca-bundle\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694014 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.693854 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-oauth-config\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694014 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.693876 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-service-ca\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694014 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.693910 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-console-config\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694014 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.693939 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvm4n\" (UniqueName: \"kubernetes.io/projected/6a759735-72a3-4679-b9d1-794aa70bad6c-kube-api-access-mvm4n\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694014 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.693968 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-oauth-serving-cert\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694014 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.694016 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-serving-cert\") pod \"6a759735-72a3-4679-b9d1-794aa70bad6c\" (UID: \"6a759735-72a3-4679-b9d1-794aa70bad6c\") " Jul 23 22:00:13.694610 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.694268 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:00:13.694610 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.694324 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:00:13.694610 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.694382 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-console-config" (OuterVolumeSpecName: "console-config") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:00:13.694610 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.694388 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:00:13.696225 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.696195 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 22:00:13.696329 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.696230 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a759735-72a3-4679-b9d1-794aa70bad6c-kube-api-access-mvm4n" (OuterVolumeSpecName: "kube-api-access-mvm4n") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "kube-api-access-mvm4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:00:13.696329 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.696262 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a759735-72a3-4679-b9d1-794aa70bad6c" (UID: "6a759735-72a3-4679-b9d1-794aa70bad6c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 22:00:13.795321 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795277 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvm4n\" (UniqueName: \"kubernetes.io/projected/6a759735-72a3-4679-b9d1-794aa70bad6c-kube-api-access-mvm4n\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:13.795321 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795314 2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-oauth-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:13.795321 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795324 2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-serving-cert\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:13.795321 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795333 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-trusted-ca-bundle\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:13.795576 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795342 2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a759735-72a3-4679-b9d1-794aa70bad6c-console-oauth-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:13.795576 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795351 2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-service-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:13.795576 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:13.795359 2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a759735-72a3-4679-b9d1-794aa70bad6c-console-config\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:14.420753 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.420719 2579 generic.go:356] "Generic (PLEG): container finished" podID="6a759735-72a3-4679-b9d1-794aa70bad6c" containerID="747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b" exitCode=0 Jul 23 22:00:14.421185 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.420766 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bd8698999-qqh5p" Jul 23 22:00:14.421185 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.420814 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd8698999-qqh5p" event={"ID":"6a759735-72a3-4679-b9d1-794aa70bad6c","Type":"ContainerDied","Data":"747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b"} Jul 23 22:00:14.421185 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.420869 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bd8698999-qqh5p" event={"ID":"6a759735-72a3-4679-b9d1-794aa70bad6c","Type":"ContainerDied","Data":"d320e0c45bc74b958d6aa57b10c83a4f0edac3c9684f388e0116c4e8feee164c"} Jul 23 22:00:14.421185 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.420891 2579 scope.go:117] "RemoveContainer" containerID="747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b" Jul 23 22:00:14.430317 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.430296 2579 scope.go:117] "RemoveContainer" containerID="747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b" Jul 23 22:00:14.430562 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:14.430544 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b\": container with ID starting with 747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b not found: ID does not exist" containerID="747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b" Jul 23 22:00:14.430623 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.430571 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b"} err="failed to get container status \"747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b\": rpc error: code = NotFound desc = could not find container \"747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b\": container with ID starting with 747844b61839e9981daa9a4c359faeb5b251245ebe9b3f41d28cc0b65ab9fc6b not found: ID does not exist" Jul 23 22:00:14.442979 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.442955 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bd8698999-qqh5p"] Jul 23 22:00:14.451806 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:14.451782 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bd8698999-qqh5p"] Jul 23 22:00:15.679320 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:15.679288 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a759735-72a3-4679-b9d1-794aa70bad6c" path="/var/lib/kubelet/pods/6a759735-72a3-4679-b9d1-794aa70bad6c/volumes" Jul 23 22:00:16.377161 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:16.377131 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:00:34.579147 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.579110 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:00:34.579644 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.579628 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a759735-72a3-4679-b9d1-794aa70bad6c" containerName="console" Jul 23 22:00:34.579698 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.579648 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a759735-72a3-4679-b9d1-794aa70bad6c" containerName="console" Jul 23 22:00:34.579747 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.579719 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a759735-72a3-4679-b9d1-794aa70bad6c" containerName="console" Jul 23 22:00:34.614075 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.614017 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:00:34.614075 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.614068 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:00:34.614312 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.614179 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.616966 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.616941 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 22:00:34.685902 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.685867 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc7e1109-a894-4150-8b08-4eeaa13fdf41-config-file\") pod \"limitador-limitador-54d6d988d5-nlhrb\" (UID: \"bc7e1109-a894-4150-8b08-4eeaa13fdf41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.685902 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.685907 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg9gs\" (UniqueName: \"kubernetes.io/projected/bc7e1109-a894-4150-8b08-4eeaa13fdf41-kube-api-access-vg9gs\") pod \"limitador-limitador-54d6d988d5-nlhrb\" (UID: \"bc7e1109-a894-4150-8b08-4eeaa13fdf41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.786786 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.786750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc7e1109-a894-4150-8b08-4eeaa13fdf41-config-file\") pod \"limitador-limitador-54d6d988d5-nlhrb\" (UID: \"bc7e1109-a894-4150-8b08-4eeaa13fdf41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.786786 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.786789 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg9gs\" (UniqueName: \"kubernetes.io/projected/bc7e1109-a894-4150-8b08-4eeaa13fdf41-kube-api-access-vg9gs\") pod \"limitador-limitador-54d6d988d5-nlhrb\" (UID: \"bc7e1109-a894-4150-8b08-4eeaa13fdf41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.787401 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.787382 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bc7e1109-a894-4150-8b08-4eeaa13fdf41-config-file\") pod \"limitador-limitador-54d6d988d5-nlhrb\" (UID: \"bc7e1109-a894-4150-8b08-4eeaa13fdf41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.795389 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.795362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg9gs\" (UniqueName: \"kubernetes.io/projected/bc7e1109-a894-4150-8b08-4eeaa13fdf41-kube-api-access-vg9gs\") pod \"limitador-limitador-54d6d988d5-nlhrb\" (UID: \"bc7e1109-a894-4150-8b08-4eeaa13fdf41\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.928965 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.928932 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:34.998148 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:34.997896 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-s5dvd"] Jul 23 22:00:35.003167 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.003140 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:35.006613 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.006321 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zr4mt\"" Jul 23 22:00:35.007824 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.007795 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5dvd"] Jul 23 22:00:35.074137 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.074112 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:00:35.076012 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:00:35.075986 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc7e1109_a894_4150_8b08_4eeaa13fdf41.slice/crio-09a6185ccddb7d0f4850e7b7fd388731712022558b310994e2a85128c8d74d05 WatchSource:0}: Error finding container 09a6185ccddb7d0f4850e7b7fd388731712022558b310994e2a85128c8d74d05: Status 404 returned error can't find the container with id 09a6185ccddb7d0f4850e7b7fd388731712022558b310994e2a85128c8d74d05 Jul 23 22:00:35.089616 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.089589 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj988\" (UniqueName: \"kubernetes.io/projected/35775e80-4166-418a-b4eb-6f1a94742147-kube-api-access-kj988\") pod \"authorino-58cd955675-s5dvd\" (UID: \"35775e80-4166-418a-b4eb-6f1a94742147\") " pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:35.190910 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.190827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj988\" (UniqueName: \"kubernetes.io/projected/35775e80-4166-418a-b4eb-6f1a94742147-kube-api-access-kj988\") pod \"authorino-58cd955675-s5dvd\" (UID: \"35775e80-4166-418a-b4eb-6f1a94742147\") " pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:35.199995 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.199969 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj988\" (UniqueName: \"kubernetes.io/projected/35775e80-4166-418a-b4eb-6f1a94742147-kube-api-access-kj988\") pod \"authorino-58cd955675-s5dvd\" (UID: \"35775e80-4166-418a-b4eb-6f1a94742147\") " pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:35.315501 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.315463 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:35.466012 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.465981 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5dvd"] Jul 23 22:00:35.470768 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:00:35.470730 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35775e80_4166_418a_b4eb_6f1a94742147.slice/crio-a14a652280c84c5fbc943518144ec3154c5aa11c2475eefff59bd08f5d79c7fa WatchSource:0}: Error finding container a14a652280c84c5fbc943518144ec3154c5aa11c2475eefff59bd08f5d79c7fa: Status 404 returned error can't find the container with id a14a652280c84c5fbc943518144ec3154c5aa11c2475eefff59bd08f5d79c7fa Jul 23 22:00:35.544001 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.543926 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5dvd" event={"ID":"35775e80-4166-418a-b4eb-6f1a94742147","Type":"ContainerStarted","Data":"a14a652280c84c5fbc943518144ec3154c5aa11c2475eefff59bd08f5d79c7fa"} Jul 23 22:00:35.546902 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:35.545667 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" event={"ID":"bc7e1109-a894-4150-8b08-4eeaa13fdf41","Type":"ContainerStarted","Data":"09a6185ccddb7d0f4850e7b7fd388731712022558b310994e2a85128c8d74d05"} Jul 23 22:00:39.572869 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:39.572834 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5dvd" event={"ID":"35775e80-4166-418a-b4eb-6f1a94742147","Type":"ContainerStarted","Data":"be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231"} Jul 23 22:00:39.574389 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:39.574368 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" event={"ID":"bc7e1109-a894-4150-8b08-4eeaa13fdf41","Type":"ContainerStarted","Data":"c134690dbd3195d429698a514fb7651a107029c5d5115cd55de380b8ac1bfa02"} Jul 23 22:00:39.574517 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:39.574502 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:00:39.591822 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:39.591773 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-s5dvd" podStartSLOduration=1.6894731360000002 podStartE2EDuration="5.591757896s" totalImagesPullingTime="3.90228476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:39.590630593 +0000 UTC m=+552.349701625" watchObservedRunningTime="2026-07-23 22:00:39.591757896 +0000 UTC m=+552.350828907" Jul 23 22:00:39.608398 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:39.608342 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" podStartSLOduration=1.32422481 podStartE2EDuration="5.608323233s" totalImagesPullingTime="4.284098423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:39.606622665 +0000 UTC m=+552.365693700" watchObservedRunningTime="2026-07-23 22:00:39.608323233 +0000 UTC m=+552.367394246" Jul 23 22:00:40.203977 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:40.203940 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5dvd"] Jul 23 22:00:41.586073 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:41.585995 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-s5dvd" podUID="35775e80-4166-418a-b4eb-6f1a94742147" containerName="authorino" containerID="cri-o://be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231" gracePeriod=30 Jul 23 22:00:41.834908 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:41.834877 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:41.957902 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:41.957874 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj988\" (UniqueName: \"kubernetes.io/projected/35775e80-4166-418a-b4eb-6f1a94742147-kube-api-access-kj988\") pod \"35775e80-4166-418a-b4eb-6f1a94742147\" (UID: \"35775e80-4166-418a-b4eb-6f1a94742147\") " Jul 23 22:00:41.959897 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:41.959865 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35775e80-4166-418a-b4eb-6f1a94742147-kube-api-access-kj988" (OuterVolumeSpecName: "kube-api-access-kj988") pod "35775e80-4166-418a-b4eb-6f1a94742147" (UID: "35775e80-4166-418a-b4eb-6f1a94742147"). InnerVolumeSpecName "kube-api-access-kj988". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:00:42.058614 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.058583 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj988\" (UniqueName: \"kubernetes.io/projected/35775e80-4166-418a-b4eb-6f1a94742147-kube-api-access-kj988\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:00:42.592016 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.591984 2579 generic.go:356] "Generic (PLEG): container finished" podID="35775e80-4166-418a-b4eb-6f1a94742147" containerID="be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231" exitCode=0 Jul 23 22:00:42.592557 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.592023 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-s5dvd" Jul 23 22:00:42.592557 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.592077 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5dvd" event={"ID":"35775e80-4166-418a-b4eb-6f1a94742147","Type":"ContainerDied","Data":"be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231"} Jul 23 22:00:42.592557 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.592113 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-s5dvd" event={"ID":"35775e80-4166-418a-b4eb-6f1a94742147","Type":"ContainerDied","Data":"a14a652280c84c5fbc943518144ec3154c5aa11c2475eefff59bd08f5d79c7fa"} Jul 23 22:00:42.592557 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.592130 2579 scope.go:117] "RemoveContainer" containerID="be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231" Jul 23 22:00:42.605402 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.605377 2579 scope.go:117] "RemoveContainer" containerID="be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231" Jul 23 22:00:42.605821 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:00:42.605794 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231\": container with ID starting with be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231 not found: ID does not exist" containerID="be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231" Jul 23 22:00:42.605921 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.605830 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231"} err="failed to get container status \"be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231\": rpc error: code = NotFound desc = could not find container \"be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231\": container with ID starting with be26afa738204eb546f818f8ebbde0f31e1debf423fce1b7e754e8e337a10231 not found: ID does not exist" Jul 23 22:00:42.619807 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.619780 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5dvd"] Jul 23 22:00:42.623437 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:42.623413 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-s5dvd"] Jul 23 22:00:43.678987 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:43.678953 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35775e80-4166-418a-b4eb-6f1a94742147" path="/var/lib/kubelet/pods/35775e80-4166-418a-b4eb-6f1a94742147/volumes" Jul 23 22:00:50.581427 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:00:50.581398 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nlhrb" Jul 23 22:01:27.623723 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:01:27.623692 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:01:27.624262 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:01:27.623989 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:01:27.630242 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:01:27.630218 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:01:27.630440 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:01:27.630428 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:01:50.976979 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:01:50.976942 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:02:09.924744 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.924705 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb"] Jul 23 22:02:09.925247 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.925232 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35775e80-4166-418a-b4eb-6f1a94742147" containerName="authorino" Jul 23 22:02:09.925300 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.925249 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="35775e80-4166-418a-b4eb-6f1a94742147" containerName="authorino" Jul 23 22:02:09.925335 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.925327 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="35775e80-4166-418a-b4eb-6f1a94742147" containerName="authorino" Jul 23 22:02:09.927371 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.927357 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:09.930166 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.930146 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 22:02:09.930424 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.930408 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 22:02:09.931628 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.931611 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-skzdc\"" Jul 23 22:02:09.931714 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.931639 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 22:02:09.941095 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:09.941071 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb"] Jul 23 22:02:10.044489 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.044457 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.044681 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.044496 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.044681 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.044534 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.044681 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.044553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.044681 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.044603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6430080-7336-471f-8872-91c55c667725-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.044844 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.044734 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r56t\" (UniqueName: \"kubernetes.io/projected/b6430080-7336-471f-8872-91c55c667725-kube-api-access-7r56t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146264 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r56t\" (UniqueName: \"kubernetes.io/projected/b6430080-7336-471f-8872-91c55c667725-kube-api-access-7r56t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146449 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146273 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146449 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146313 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146449 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146364 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146449 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146449 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146414 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6430080-7336-471f-8872-91c55c667725-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146813 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146784 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146885 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146793 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.146885 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.146827 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.148646 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.148627 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6430080-7336-471f-8872-91c55c667725-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.148931 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.148913 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6430080-7336-471f-8872-91c55c667725-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.155579 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.155552 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r56t\" (UniqueName: \"kubernetes.io/projected/b6430080-7336-471f-8872-91c55c667725-kube-api-access-7r56t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb\" (UID: \"b6430080-7336-471f-8872-91c55c667725\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.238546 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.238456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:10.376489 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.376467 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb"] Jul 23 22:02:10.378382 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:02:10.378355 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6430080_7336_471f_8872_91c55c667725.slice/crio-12c8326c1cc56517dd1c889ee7e618618c087e547f408295fa7c4eda67aa08c5 WatchSource:0}: Error finding container 12c8326c1cc56517dd1c889ee7e618618c087e547f408295fa7c4eda67aa08c5: Status 404 returned error can't find the container with id 12c8326c1cc56517dd1c889ee7e618618c087e547f408295fa7c4eda67aa08c5 Jul 23 22:02:10.380098 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.380079 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 22:02:10.512977 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:10.512899 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:02:11.066615 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:11.066513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" event={"ID":"b6430080-7336-471f-8872-91c55c667725","Type":"ContainerStarted","Data":"12c8326c1cc56517dd1c889ee7e618618c087e547f408295fa7c4eda67aa08c5"} Jul 23 22:02:12.073969 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:12.073930 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" event={"ID":"b6430080-7336-471f-8872-91c55c667725","Type":"ContainerStarted","Data":"ae76d159138efdfa6f9f5c8ab932d157cbfe1740ec7d2639267c02a5cf3a13b1"} Jul 23 22:02:12.099020 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:12.098971 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" podStartSLOduration=1.969525183 podStartE2EDuration="3.098953635s" totalImagesPullingTime="1.129428452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:02:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:02:12.09673308 +0000 UTC m=+644.855804092" watchObservedRunningTime="2026-07-23 22:02:12.098953635 +0000 UTC m=+644.858024646" Jul 23 22:02:20.239041 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:20.239001 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:20.239515 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:20.239092 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:20.251839 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:20.251814 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:21.133953 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:21.133924 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb" Jul 23 22:02:25.812941 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:25.812907 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:02:37.317816 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:37.317780 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:02:48.797833 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.797798 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4"] Jul 23 22:02:48.805785 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.805757 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.809116 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.809093 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 23 22:02:48.813162 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.813138 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4"] Jul 23 22:02:48.897123 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.897090 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.897296 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.897144 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxnrq\" (UniqueName: \"kubernetes.io/projected/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-kube-api-access-lxnrq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.897296 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.897241 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.897296 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.897274 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.897423 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.897307 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.897423 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.897330 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.997910 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.997876 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998128 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.997931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998128 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.997967 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxnrq\" (UniqueName: \"kubernetes.io/projected/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-kube-api-access-lxnrq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998128 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.998009 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998128 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.998097 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998404 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.998164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998451 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.998416 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998488 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.998472 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:48.998537 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:48.998489 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:49.000298 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:49.000270 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:49.000441 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:49.000424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:49.006367 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:49.006350 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxnrq\" (UniqueName: \"kubernetes.io/projected/f5f7ff3c-6bf4-4440-af13-21e7bd457ce2-kube-api-access-lxnrq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4\" (UID: \"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:49.119223 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:49.119190 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:49.265838 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:49.265815 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4"] Jul 23 22:02:49.268690 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:02:49.268654 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f7ff3c_6bf4_4440_af13_21e7bd457ce2.slice/crio-0bd9aee5aeb106c95ababb076b1c6e36ca63cdd53344e2d8f16fe5cbb93c2c36 WatchSource:0}: Error finding container 0bd9aee5aeb106c95ababb076b1c6e36ca63cdd53344e2d8f16fe5cbb93c2c36: Status 404 returned error can't find the container with id 0bd9aee5aeb106c95ababb076b1c6e36ca63cdd53344e2d8f16fe5cbb93c2c36 Jul 23 22:02:49.279037 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:49.279007 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" event={"ID":"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2","Type":"ContainerStarted","Data":"0bd9aee5aeb106c95ababb076b1c6e36ca63cdd53344e2d8f16fe5cbb93c2c36"} Jul 23 22:02:50.285841 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:50.285807 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" event={"ID":"f5f7ff3c-6bf4-4440-af13-21e7bd457ce2","Type":"ContainerStarted","Data":"429ab732b99f78088b47fc8c1012e0f4917b9a6dee991568d799946421783661"} Jul 23 22:02:50.309828 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:50.309765 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" podStartSLOduration=2.106835048 podStartE2EDuration="2.309743329s" totalImagesPullingTime="202.908281ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:02:50.30946841 +0000 UTC m=+683.068539422" watchObservedRunningTime="2026-07-23 22:02:50.309743329 +0000 UTC m=+683.068814341" Jul 23 22:02:50.318756 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:50.318721 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:02:57.035260 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:57.035223 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:02:59.119542 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:59.119511 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:59.119983 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:59.119611 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:59.131691 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:59.131670 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:02:59.349068 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:02:59.349027 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4" Jul 23 22:03:08.615563 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:03:08.615531 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:04:05.456762 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.456729 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-65696fb76-kbrw9"] Jul 23 22:04:05.460520 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.460498 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.464207 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.464184 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 23 22:04:05.464310 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.464186 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zr4mt\"" Jul 23 22:04:05.468599 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.468577 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65696fb76-kbrw9"] Jul 23 22:04:05.532744 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.532712 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f-tls-cert\") pod \"authorino-65696fb76-kbrw9\" (UID: \"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f\") " pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.532926 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.532782 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmmzn\" (UniqueName: \"kubernetes.io/projected/aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f-kube-api-access-tmmzn\") pod \"authorino-65696fb76-kbrw9\" (UID: \"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f\") " pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.634126 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.634089 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmmzn\" (UniqueName: \"kubernetes.io/projected/aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f-kube-api-access-tmmzn\") pod \"authorino-65696fb76-kbrw9\" (UID: \"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f\") " pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.634274 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.634166 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f-tls-cert\") pod \"authorino-65696fb76-kbrw9\" (UID: \"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f\") " pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.636506 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.636483 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f-tls-cert\") pod \"authorino-65696fb76-kbrw9\" (UID: \"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f\") " pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.642757 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.642738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmmzn\" (UniqueName: \"kubernetes.io/projected/aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f-kube-api-access-tmmzn\") pod \"authorino-65696fb76-kbrw9\" (UID: \"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f\") " pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.770700 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.770623 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-65696fb76-kbrw9" Jul 23 22:04:05.925333 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:05.925307 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-65696fb76-kbrw9"] Jul 23 22:04:05.927533 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:04:05.927505 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaf5cea1_3a4a_4d11_bdf6_c6ef02f3660f.slice/crio-4ab77ec12fe6045f6dffbf878bbd468bf077f5319ae4c95b7e60707c85c3f2b6 WatchSource:0}: Error finding container 4ab77ec12fe6045f6dffbf878bbd468bf077f5319ae4c95b7e60707c85c3f2b6: Status 404 returned error can't find the container with id 4ab77ec12fe6045f6dffbf878bbd468bf077f5319ae4c95b7e60707c85c3f2b6 Jul 23 22:04:06.700593 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:06.700502 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65696fb76-kbrw9" event={"ID":"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f","Type":"ContainerStarted","Data":"e2919af9089ef9e62376b89b11ae84a0af0a9ff2fc0ce86a71cb78234bed3b10"} Jul 23 22:04:06.700593 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:06.700544 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-65696fb76-kbrw9" event={"ID":"aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f","Type":"ContainerStarted","Data":"4ab77ec12fe6045f6dffbf878bbd468bf077f5319ae4c95b7e60707c85c3f2b6"} Jul 23 22:04:36.922722 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:36.922662 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-65696fb76-kbrw9" podStartSLOduration=31.450349853 podStartE2EDuration="31.922643553s" totalImagesPullingTime="472.2937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:04:06.720352359 +0000 UTC m=+759.479423370" watchObservedRunningTime="2026-07-23 22:04:36.922643553 +0000 UTC m=+789.681714626" Jul 23 22:04:36.923216 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:36.923194 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:04:46.518196 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:46.518161 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:04:57.122029 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:04:57.121951 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:06:27.658559 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:06:27.658520 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:06:27.663237 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:06:27.663215 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:06:27.665741 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:06:27.665719 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:06:27.669756 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:06:27.669742 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:06:37.328146 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:06:37.328111 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:06:48.716086 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:06:48.715299 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:07:24.511700 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:07:24.511663 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:07:34.221436 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:07:34.221400 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:07:44.010219 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:07:44.010173 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:07:51.708617 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:07:51.708544 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:08:03.057291 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:08:03.057249 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:08:14.716828 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:08:14.716780 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:08:25.615078 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:08:25.615025 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:08:35.716807 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:08:35.716759 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:08:58.215971 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:08:58.215935 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:09:10.611126 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:09:10.611093 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:09:33.920732 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:09:33.920697 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:09:44.710044 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:09:44.710010 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:09:53.034983 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:09:53.034941 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:10:02.115620 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:10:02.115579 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:10:12.117826 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:10:12.117787 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:10:21.908365 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:10:21.908327 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:10:31.513010 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:10:31.512972 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:10:40.907102 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:10:40.907066 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:11:07.238347 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:07.238258 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458"] Jul 23 22:11:07.238888 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:07.238534 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" podUID="08720b79-3f36-4808-a4c4-d8e580ba6a06" containerName="manager" containerID="cri-o://31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768" gracePeriod=10 Jul 23 22:11:09.501485 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.501460 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:11:09.564435 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.564354 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08720b79-3f36-4808-a4c4-d8e580ba6a06-extensions-socket-volume\") pod \"08720b79-3f36-4808-a4c4-d8e580ba6a06\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " Jul 23 22:11:09.564594 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.564466 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmvvw\" (UniqueName: \"kubernetes.io/projected/08720b79-3f36-4808-a4c4-d8e580ba6a06-kube-api-access-kmvvw\") pod \"08720b79-3f36-4808-a4c4-d8e580ba6a06\" (UID: \"08720b79-3f36-4808-a4c4-d8e580ba6a06\") " Jul 23 22:11:09.564814 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.564780 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08720b79-3f36-4808-a4c4-d8e580ba6a06-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "08720b79-3f36-4808-a4c4-d8e580ba6a06" (UID: "08720b79-3f36-4808-a4c4-d8e580ba6a06"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:11:09.566437 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.566408 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08720b79-3f36-4808-a4c4-d8e580ba6a06-kube-api-access-kmvvw" (OuterVolumeSpecName: "kube-api-access-kmvvw") pod "08720b79-3f36-4808-a4c4-d8e580ba6a06" (UID: "08720b79-3f36-4808-a4c4-d8e580ba6a06"). InnerVolumeSpecName "kube-api-access-kmvvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:11:09.665338 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.665304 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmvvw\" (UniqueName: \"kubernetes.io/projected/08720b79-3f36-4808-a4c4-d8e580ba6a06-kube-api-access-kmvvw\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:11:09.665338 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:09.665334 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/08720b79-3f36-4808-a4c4-d8e580ba6a06-extensions-socket-volume\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:11:10.104373 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.104338 2579 generic.go:356] "Generic (PLEG): container finished" podID="08720b79-3f36-4808-a4c4-d8e580ba6a06" containerID="31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768" exitCode=0 Jul 23 22:11:10.104574 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.104391 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" Jul 23 22:11:10.104574 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.104424 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" event={"ID":"08720b79-3f36-4808-a4c4-d8e580ba6a06","Type":"ContainerDied","Data":"31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768"} Jul 23 22:11:10.104574 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.104478 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458" event={"ID":"08720b79-3f36-4808-a4c4-d8e580ba6a06","Type":"ContainerDied","Data":"e6ef20dccadc2569bf4ad0912ed2cb6c0497cc3372286db6116ace095845c9fb"} Jul 23 22:11:10.104574 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.104502 2579 scope.go:117] "RemoveContainer" containerID="31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768" Jul 23 22:11:10.118317 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.118293 2579 scope.go:117] "RemoveContainer" containerID="31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768" Jul 23 22:11:10.118625 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:11:10.118601 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768\": container with ID starting with 31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768 not found: ID does not exist" containerID="31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768" Jul 23 22:11:10.118702 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.118633 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768"} err="failed to get container status \"31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768\": rpc error: code = NotFound desc = could not find container \"31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768\": container with ID starting with 31dbcdcbdaa8d899dddcf02ec85d55fb75af1cc594d644b5e35b8247173c3768 not found: ID does not exist" Jul 23 22:11:10.124613 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.124589 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458"] Jul 23 22:11:10.128017 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:10.127993 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5w458"] Jul 23 22:11:11.679108 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:11.679078 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08720b79-3f36-4808-a4c4-d8e580ba6a06" path="/var/lib/kubelet/pods/08720b79-3f36-4808-a4c4-d8e580ba6a06/volumes" Jul 23 22:11:27.693892 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:27.693865 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:11:27.698543 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:27.698519 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:11:27.700853 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:27.700834 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:11:27.705018 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:11:27.705002 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:12:08.434323 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.434288 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg"] Jul 23 22:12:08.436655 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.434719 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08720b79-3f36-4808-a4c4-d8e580ba6a06" containerName="manager" Jul 23 22:12:08.436655 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.434729 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08720b79-3f36-4808-a4c4-d8e580ba6a06" containerName="manager" Jul 23 22:12:08.436655 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.434797 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08720b79-3f36-4808-a4c4-d8e580ba6a06" containerName="manager" Jul 23 22:12:08.437711 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.437694 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.440769 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.440742 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hkd2b\"" Jul 23 22:12:08.451682 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.451657 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg"] Jul 23 22:12:08.510363 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.510318 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbmcz\" (UniqueName: \"kubernetes.io/projected/b4e38394-8399-4bd4-92ec-146cfba6cc5c-kube-api-access-mbmcz\") pod \"kuadrant-operator-controller-manager-86878c769f-mmqvg\" (UID: \"b4e38394-8399-4bd4-92ec-146cfba6cc5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.510363 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.510371 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4e38394-8399-4bd4-92ec-146cfba6cc5c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mmqvg\" (UID: \"b4e38394-8399-4bd4-92ec-146cfba6cc5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.611944 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.611906 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4e38394-8399-4bd4-92ec-146cfba6cc5c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mmqvg\" (UID: \"b4e38394-8399-4bd4-92ec-146cfba6cc5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.612167 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.612086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbmcz\" (UniqueName: \"kubernetes.io/projected/b4e38394-8399-4bd4-92ec-146cfba6cc5c-kube-api-access-mbmcz\") pod \"kuadrant-operator-controller-manager-86878c769f-mmqvg\" (UID: \"b4e38394-8399-4bd4-92ec-146cfba6cc5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.612401 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.612381 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4e38394-8399-4bd4-92ec-146cfba6cc5c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mmqvg\" (UID: \"b4e38394-8399-4bd4-92ec-146cfba6cc5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.621034 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.621003 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbmcz\" (UniqueName: \"kubernetes.io/projected/b4e38394-8399-4bd4-92ec-146cfba6cc5c-kube-api-access-mbmcz\") pod \"kuadrant-operator-controller-manager-86878c769f-mmqvg\" (UID: \"b4e38394-8399-4bd4-92ec-146cfba6cc5c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.748122 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.747990 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:08.894356 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:08.894329 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg"] Jul 23 22:12:08.896590 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:12:08.896565 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e38394_8399_4bd4_92ec_146cfba6cc5c.slice/crio-833fe4564cb8bf1a0ca9bbcf34086178631e9efa8e41950d03c8cea7c2d2fe93 WatchSource:0}: Error finding container 833fe4564cb8bf1a0ca9bbcf34086178631e9efa8e41950d03c8cea7c2d2fe93: Status 404 returned error can't find the container with id 833fe4564cb8bf1a0ca9bbcf34086178631e9efa8e41950d03c8cea7c2d2fe93 Jul 23 22:12:09.424989 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:09.424956 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" event={"ID":"b4e38394-8399-4bd4-92ec-146cfba6cc5c","Type":"ContainerStarted","Data":"41c9d8e0802cbe4a4dfa4cbadbfc8716fc5fbe5d289f3d0d960760d49002a0b0"} Jul 23 22:12:09.425193 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:09.425008 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" event={"ID":"b4e38394-8399-4bd4-92ec-146cfba6cc5c","Type":"ContainerStarted","Data":"833fe4564cb8bf1a0ca9bbcf34086178631e9efa8e41950d03c8cea7c2d2fe93"} Jul 23 22:12:09.425193 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:09.425080 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:09.448562 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:09.448510 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" podStartSLOduration=1.448494876 podStartE2EDuration="1.448494876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:12:09.445213098 +0000 UTC m=+1242.204284131" watchObservedRunningTime="2026-07-23 22:12:09.448494876 +0000 UTC m=+1242.207565887" Jul 23 22:12:20.432866 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:20.432836 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mmqvg" Jul 23 22:12:29.515406 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:29.515372 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:12:39.011248 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:39.011215 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:12:52.611931 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:12:52.611898 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:13:03.214755 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:13:03.214722 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:13:12.815829 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:13:12.815793 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:13:20.915857 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:13:20.915822 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:13:32.615206 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:13:32.615168 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:13:42.705067 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:13:42.705023 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:13:51.916360 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:13:51.916279 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:14:01.711549 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:14:01.711515 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:14:14.813792 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:14:14.813757 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:14:24.116466 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:14:24.116433 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:14:44.511714 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:14:44.511677 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:14:52.319363 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:14:52.319326 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:15:04.128578 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:15:04.128537 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:15:21.208786 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:15:21.208752 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:16:04.523891 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:04.523853 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:16:17.910005 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:17.909972 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:16:27.728722 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:27.728690 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:16:27.734490 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:27.734469 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:16:27.735340 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:27.735318 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:16:27.740847 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:27.740829 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:16:27.945819 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:27.945788 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:16:40.714011 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:40.713974 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:16:55.146084 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:16:55.145992 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:17:04.814047 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:17:04.814014 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:17:18.477499 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:17:18.477465 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:17:28.634643 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:17:28.634599 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:17:39.314251 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:17:39.314218 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:17:50.918109 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:17:50.918077 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:18:00.728227 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:18:00.728187 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:18:13.812843 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:18:13.812799 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:18:25.309136 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:18:25.309101 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:18:33.317308 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:18:33.317272 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:18:45.224725 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:18:45.224682 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:18:57.630351 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:18:57.630319 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:19:11.317736 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:19:11.317701 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:19:20.725095 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:19:20.725031 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:19:43.113645 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:19:43.113606 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:19:54.015972 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:19:54.015893 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:20:29.820534 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:20:29.820498 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:20:41.015677 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:20:41.015642 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:21:16.215340 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:21:16.215305 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:21:27.764695 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:21:27.764662 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:21:27.771542 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:21:27.771514 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:21:27.772070 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:21:27.772036 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:21:27.778261 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:21:27.778245 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:21:28.611131 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:21:28.611098 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:22:00.308832 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:22:00.308794 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:22:13.720323 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:22:13.720290 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:22:45.309538 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:22:45.309506 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:24:15.719504 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:24:15.719470 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:24:24.717486 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:24:24.717449 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:24:37.643071 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:24:37.643013 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:24:58.119874 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:24:58.119833 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:25:37.618247 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:25:37.618211 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:25:55.118394 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:25:55.118316 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:26:12.416983 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:12.416951 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:26:27.799330 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:27.799298 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:26:27.805560 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:27.805539 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:26:27.807193 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:27.807173 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:26:27.813312 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:27.813293 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:26:32.718358 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:32.718329 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:26:44.120212 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:26:44.120179 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:27:34.140411 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:27:34.140370 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:27:44.524491 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:27:44.524448 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:28:37.327952 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:28:37.327908 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:28:49.410147 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:28:49.410047 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:28:59.208475 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:28:59.208444 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:29:23.214294 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:29:23.214263 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:29:37.311132 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:29:37.311096 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:30:00.028400 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.028364 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps"] Jul 23 22:30:00.031981 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.031964 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:00.034509 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.034491 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 22:30:00.034613 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.034539 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lq4mm\"" Jul 23 22:30:00.039451 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.039426 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps"] Jul 23 22:30:00.119354 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.119316 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1bfc3ca-479d-4ba4-baa0-53d798d20b99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps\" (UID: \"e1bfc3ca-479d-4ba4-baa0-53d798d20b99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:00.142922 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.142885 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq"] Jul 23 22:30:00.147226 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.147207 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl"] Jul 23 22:30:00.147337 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.147320 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" Jul 23 22:30:00.150732 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.150715 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" Jul 23 22:30:00.154195 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.154171 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq"] Jul 23 22:30:00.158236 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.158214 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl"] Jul 23 22:30:00.167603 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.167577 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" Jul 23 22:30:00.173865 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.173848 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" Jul 23 22:30:00.221086 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.220033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1bfc3ca-479d-4ba4-baa0-53d798d20b99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps\" (UID: \"e1bfc3ca-479d-4ba4-baa0-53d798d20b99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:00.221086 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.220925 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1bfc3ca-479d-4ba4-baa0-53d798d20b99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps\" (UID: \"e1bfc3ca-479d-4ba4-baa0-53d798d20b99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:00.324303 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.324273 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq"] Jul 23 22:30:00.327340 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:30:00.327306 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb5c275e_dad9_4c61_bd2b_ec98dd3af010.slice/crio-2666bd922dd2f112d56dbaf77afcd8093e63009708be8b9833e527ae2f3b3fac WatchSource:0}: Error finding container 2666bd922dd2f112d56dbaf77afcd8093e63009708be8b9833e527ae2f3b3fac: Status 404 returned error can't find the container with id 2666bd922dd2f112d56dbaf77afcd8093e63009708be8b9833e527ae2f3b3fac Jul 23 22:30:00.330154 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.330120 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 22:30:00.344610 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.344586 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:00.358785 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.357819 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl"] Jul 23 22:30:00.360003 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:30:00.359969 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b0eeaa_d2ef_4c78_8777_f127ce757142.slice/crio-3b130cb680b952c1237a03f464b23cc6e9b88663229f49b86a4a076abe0abf4f WatchSource:0}: Error finding container 3b130cb680b952c1237a03f464b23cc6e9b88663229f49b86a4a076abe0abf4f: Status 404 returned error can't find the container with id 3b130cb680b952c1237a03f464b23cc6e9b88663229f49b86a4a076abe0abf4f Jul 23 22:30:00.451870 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.451840 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" event={"ID":"c6b0eeaa-d2ef-4c78-8777-f127ce757142","Type":"ContainerStarted","Data":"3b130cb680b952c1237a03f464b23cc6e9b88663229f49b86a4a076abe0abf4f"} Jul 23 22:30:00.453372 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.453340 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" event={"ID":"cb5c275e-dad9-4c61-bd2b-ec98dd3af010","Type":"ContainerStarted","Data":"2666bd922dd2f112d56dbaf77afcd8093e63009708be8b9833e527ae2f3b3fac"} Jul 23 22:30:00.492855 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:00.492832 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps"] Jul 23 22:30:00.495167 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:30:00.495135 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1bfc3ca_479d_4ba4_baa0_53d798d20b99.slice/crio-7e249f3561ce64348b9997677883297ccac4820abc1120bd90c049d4be2172eb WatchSource:0}: Error finding container 7e249f3561ce64348b9997677883297ccac4820abc1120bd90c049d4be2172eb: Status 404 returned error can't find the container with id 7e249f3561ce64348b9997677883297ccac4820abc1120bd90c049d4be2172eb Jul 23 22:30:01.460549 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:01.460513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" event={"ID":"e1bfc3ca-479d-4ba4-baa0-53d798d20b99","Type":"ContainerStarted","Data":"7e249f3561ce64348b9997677883297ccac4820abc1120bd90c049d4be2172eb"} Jul 23 22:30:04.484232 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:04.484196 2579 generic.go:356] "Generic (PLEG): container finished" podID="cb5c275e-dad9-4c61-bd2b-ec98dd3af010" containerID="0ee579b42e97d00a90d299f76fa6d6c2d12f083cc4106c561276cf81ba73ff2f" exitCode=0 Jul 23 22:30:04.484777 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:04.484276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" event={"ID":"cb5c275e-dad9-4c61-bd2b-ec98dd3af010","Type":"ContainerDied","Data":"0ee579b42e97d00a90d299f76fa6d6c2d12f083cc4106c561276cf81ba73ff2f"} Jul 23 22:30:04.485775 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:04.485752 2579 generic.go:356] "Generic (PLEG): container finished" podID="c6b0eeaa-d2ef-4c78-8777-f127ce757142" containerID="2a35c199c50fecf70593e615850d8f77c49aebb543a11bc7acbc0d8590a2ec18" exitCode=0 Jul 23 22:30:04.485890 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:04.485780 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" event={"ID":"c6b0eeaa-d2ef-4c78-8777-f127ce757142","Type":"ContainerDied","Data":"2a35c199c50fecf70593e615850d8f77c49aebb543a11bc7acbc0d8590a2ec18"} Jul 23 22:30:04.487239 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:04.487218 2579 generic.go:356] "Generic (PLEG): container finished" podID="e1bfc3ca-479d-4ba4-baa0-53d798d20b99" containerID="caa28fff12c43845c8b6acc02dd2512b90fdd519c63616310c8e1636ce0a2811" exitCode=0 Jul 23 22:30:04.487341 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:04.487276 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" event={"ID":"e1bfc3ca-479d-4ba4-baa0-53d798d20b99","Type":"ContainerDied","Data":"caa28fff12c43845c8b6acc02dd2512b90fdd519c63616310c8e1636ce0a2811"} Jul 23 22:30:05.657761 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:05.657727 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:05.699581 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:05.699552 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" Jul 23 22:30:05.703418 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:05.703395 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" Jul 23 22:30:05.778556 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:05.778480 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1bfc3ca-479d-4ba4-baa0-53d798d20b99-maas-api-service-ca\") pod \"e1bfc3ca-479d-4ba4-baa0-53d798d20b99\" (UID: \"e1bfc3ca-479d-4ba4-baa0-53d798d20b99\") " Jul 23 22:30:05.778919 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:05.778893 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1bfc3ca-479d-4ba4-baa0-53d798d20b99-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e1bfc3ca-479d-4ba4-baa0-53d798d20b99" (UID: "e1bfc3ca-479d-4ba4-baa0-53d798d20b99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:30:05.879654 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:05.879621 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1bfc3ca-479d-4ba4-baa0-53d798d20b99-maas-api-service-ca\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:30:06.500251 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.500217 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" Jul 23 22:30:06.500415 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.500266 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq" event={"ID":"cb5c275e-dad9-4c61-bd2b-ec98dd3af010","Type":"ContainerDied","Data":"2666bd922dd2f112d56dbaf77afcd8093e63009708be8b9833e527ae2f3b3fac"} Jul 23 22:30:06.500415 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.500312 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2666bd922dd2f112d56dbaf77afcd8093e63009708be8b9833e527ae2f3b3fac" Jul 23 22:30:06.501718 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.501697 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" Jul 23 22:30:06.501847 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.501718 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29747430-knmnl" event={"ID":"c6b0eeaa-d2ef-4c78-8777-f127ce757142","Type":"ContainerDied","Data":"3b130cb680b952c1237a03f464b23cc6e9b88663229f49b86a4a076abe0abf4f"} Jul 23 22:30:06.501847 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.501763 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b130cb680b952c1237a03f464b23cc6e9b88663229f49b86a4a076abe0abf4f" Jul 23 22:30:06.503136 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.503119 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" Jul 23 22:30:06.503253 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.503166 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps" event={"ID":"e1bfc3ca-479d-4ba4-baa0-53d798d20b99","Type":"ContainerDied","Data":"7e249f3561ce64348b9997677883297ccac4820abc1120bd90c049d4be2172eb"} Jul 23 22:30:06.503253 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.503206 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e249f3561ce64348b9997677883297ccac4820abc1120bd90c049d4be2172eb" Jul 23 22:30:06.617003 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:06.616969 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:30:10.490162 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:10.490121 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq"] Jul 23 22:30:10.493067 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:10.493023 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-8ba6ab-29747430-cmpbq"] Jul 23 22:30:11.679397 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:11.679360 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb5c275e-dad9-4c61-bd2b-ec98dd3af010" path="/var/lib/kubelet/pods/cb5c275e-dad9-4c61-bd2b-ec98dd3af010/volumes" Jul 23 22:30:15.141965 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:15.141928 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps"] Jul 23 22:30:15.143899 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:15.143878 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-8ba6ab-ddhps"] Jul 23 22:30:15.680423 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:15.680392 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1bfc3ca-479d-4ba4-baa0-53d798d20b99" path="/var/lib/kubelet/pods/e1bfc3ca-479d-4ba4-baa0-53d798d20b99/volumes" Jul 23 22:30:16.412583 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:16.412549 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:30:36.117046 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:36.117014 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:30:55.715613 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:30:55.715577 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:31:11.516936 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:11.516903 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:31:27.833945 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:27.833920 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:31:27.840829 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:27.840803 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:31:27.845918 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:27.845899 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:31:27.851997 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:27.851978 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:31:30.917460 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:30.917412 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:31:57.309371 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:31:57.309294 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:32:12.526339 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:32:12.526292 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:32:23.817848 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:32:23.817810 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:32:41.018911 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:32:41.018877 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:33:02.408113 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:33:02.408079 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:33:30.022941 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:33:30.022903 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:33:48.310795 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:33:48.310764 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:34:04.010308 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:34:04.010270 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:34:20.507203 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:34:20.507168 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:34:37.826192 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:34:37.826156 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:34:55.613691 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:34:55.613611 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:35:13.514543 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:35:13.514505 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:35:31.019169 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:35:31.019133 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:36:11.415795 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:11.415762 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:36:27.874788 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:27.874759 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:36:27.884645 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:27.884620 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:36:27.897930 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:27.897903 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:36:27.904400 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:27.904381 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlb2q_1914f60a-ab07-4a57-8c3e-05cdcc59407c/ovn-acl-logging/0.log" Jul 23 22:36:27.983549 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:27.983466 2579 scope.go:117] "RemoveContainer" containerID="0ee579b42e97d00a90d299f76fa6d6c2d12f083cc4106c561276cf81ba73ff2f" Jul 23 22:36:27.992126 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:27.992101 2579 scope.go:117] "RemoveContainer" containerID="caa28fff12c43845c8b6acc02dd2512b90fdd519c63616310c8e1636ce0a2811" Jul 23 22:36:34.122155 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:34.122119 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:36:52.406951 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:36:52.406915 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:37:10.608334 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:37:10.608303 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:37:25.107670 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:37:25.107637 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:37:43.318359 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:37:43.318322 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:38:10.333994 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:10.333904 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:38:26.219925 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:26.219890 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:38:34.454745 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.454716 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7721-139888552700592"] Jul 23 22:38:34.455181 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455166 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1bfc3ca-479d-4ba4-baa0-53d798d20b99" containerName="revoke-keys" Jul 23 22:38:34.455181 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455181 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1bfc3ca-479d-4ba4-baa0-53d798d20b99" containerName="revoke-keys" Jul 23 22:38:34.455257 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455189 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb5c275e-dad9-4c61-bd2b-ec98dd3af010" containerName="cleanup" Jul 23 22:38:34.455257 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455194 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb5c275e-dad9-4c61-bd2b-ec98dd3af010" containerName="cleanup" Jul 23 22:38:34.455257 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455212 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6b0eeaa-d2ef-4c78-8777-f127ce757142" containerName="cleanup" Jul 23 22:38:34.455257 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455217 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6b0eeaa-d2ef-4c78-8777-f127ce757142" containerName="cleanup" Jul 23 22:38:34.455386 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455279 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb5c275e-dad9-4c61-bd2b-ec98dd3af010" containerName="cleanup" Jul 23 22:38:34.455386 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455288 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1bfc3ca-479d-4ba4-baa0-53d798d20b99" containerName="revoke-keys" Jul 23 22:38:34.455386 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.455297 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6b0eeaa-d2ef-4c78-8777-f127ce757142" containerName="cleanup" Jul 23 22:38:34.458557 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.458541 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:34.461676 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.461651 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bqb75\"" Jul 23 22:38:34.468154 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.468131 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552700592"] Jul 23 22:38:34.507205 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.507178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tw9s\" (UniqueName: \"kubernetes.io/projected/c98bf967-b888-43a5-aa40-339c6ec01e0e-kube-api-access-2tw9s\") pod \"test-curl-7721-139888552700592\" (UID: \"c98bf967-b888-43a5-aa40-339c6ec01e0e\") " pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:34.608089 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.608047 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tw9s\" (UniqueName: \"kubernetes.io/projected/c98bf967-b888-43a5-aa40-339c6ec01e0e-kube-api-access-2tw9s\") pod \"test-curl-7721-139888552700592\" (UID: \"c98bf967-b888-43a5-aa40-339c6ec01e0e\") " pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:34.615844 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.615817 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tw9s\" (UniqueName: \"kubernetes.io/projected/c98bf967-b888-43a5-aa40-339c6ec01e0e-kube-api-access-2tw9s\") pod \"test-curl-7721-139888552700592\" (UID: \"c98bf967-b888-43a5-aa40-339c6ec01e0e\") " pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:34.769360 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.769274 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:34.905736 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.905712 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552700592"] Jul 23 22:38:34.908644 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:38:34.908616 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc98bf967_b888_43a5_aa40_339c6ec01e0e.slice/crio-d55518b4e0f4422fceff6ac7489f1366bf32fa93b07c6ee53f8462f73f3b4af5 WatchSource:0}: Error finding container d55518b4e0f4422fceff6ac7489f1366bf32fa93b07c6ee53f8462f73f3b4af5: Status 404 returned error can't find the container with id d55518b4e0f4422fceff6ac7489f1366bf32fa93b07c6ee53f8462f73f3b4af5 Jul 23 22:38:34.910046 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:34.910024 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 22:38:35.441780 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:35.441736 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552700592" event={"ID":"c98bf967-b888-43a5-aa40-339c6ec01e0e","Type":"ContainerStarted","Data":"d55518b4e0f4422fceff6ac7489f1366bf32fa93b07c6ee53f8462f73f3b4af5"} Jul 23 22:38:36.448365 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:36.448330 2579 generic.go:356] "Generic (PLEG): container finished" podID="c98bf967-b888-43a5-aa40-339c6ec01e0e" containerID="85b23721145b1418d434a83e67f91260e3b0c83e1791b506b627b4813b76afc7" exitCode=0 Jul 23 22:38:36.448739 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:36.448373 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552700592" event={"ID":"c98bf967-b888-43a5-aa40-339c6ec01e0e","Type":"ContainerDied","Data":"85b23721145b1418d434a83e67f91260e3b0c83e1791b506b627b4813b76afc7"} Jul 23 22:38:37.599555 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.599529 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:37.636739 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.636704 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tw9s\" (UniqueName: \"kubernetes.io/projected/c98bf967-b888-43a5-aa40-339c6ec01e0e-kube-api-access-2tw9s\") pod \"c98bf967-b888-43a5-aa40-339c6ec01e0e\" (UID: \"c98bf967-b888-43a5-aa40-339c6ec01e0e\") " Jul 23 22:38:37.638799 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.638771 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c98bf967-b888-43a5-aa40-339c6ec01e0e-kube-api-access-2tw9s" (OuterVolumeSpecName: "kube-api-access-2tw9s") pod "c98bf967-b888-43a5-aa40-339c6ec01e0e" (UID: "c98bf967-b888-43a5-aa40-339c6ec01e0e"). InnerVolumeSpecName "kube-api-access-2tw9s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:38:37.657231 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.657212 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7721-139888552700592_c98bf967-b888-43a5-aa40-339c6ec01e0e/test-curl-7721-139888552700592/0.log" Jul 23 22:38:37.700665 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.700639 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7721-139888552700592"] Jul 23 22:38:37.714022 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.713960 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7721-139888552700592"] Jul 23 22:38:37.738109 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.738082 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tw9s\" (UniqueName: \"kubernetes.io/projected/c98bf967-b888-43a5-aa40-339c6ec01e0e-kube-api-access-2tw9s\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:38:37.834664 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.834630 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7721-139888552851376"] Jul 23 22:38:37.835189 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.835173 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c98bf967-b888-43a5-aa40-339c6ec01e0e" containerName="test-curl-7721-139888552700592" Jul 23 22:38:37.835310 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.835193 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c98bf967-b888-43a5-aa40-339c6ec01e0e" containerName="test-curl-7721-139888552700592" Jul 23 22:38:37.835310 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.835304 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c98bf967-b888-43a5-aa40-339c6ec01e0e" containerName="test-curl-7721-139888552700592" Jul 23 22:38:37.838399 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.838371 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:37.845400 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.845379 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552851376"] Jul 23 22:38:37.940112 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:37.940080 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dc8b\" (UniqueName: \"kubernetes.io/projected/34f4a295-8c76-49e7-b15d-8e82c50c80c0-kube-api-access-4dc8b\") pod \"test-curl-7721-139888552851376\" (UID: \"34f4a295-8c76-49e7-b15d-8e82c50c80c0\") " pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:38.041178 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.041086 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dc8b\" (UniqueName: \"kubernetes.io/projected/34f4a295-8c76-49e7-b15d-8e82c50c80c0-kube-api-access-4dc8b\") pod \"test-curl-7721-139888552851376\" (UID: \"34f4a295-8c76-49e7-b15d-8e82c50c80c0\") " pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:38.051215 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.051186 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dc8b\" (UniqueName: \"kubernetes.io/projected/34f4a295-8c76-49e7-b15d-8e82c50c80c0-kube-api-access-4dc8b\") pod \"test-curl-7721-139888552851376\" (UID: \"34f4a295-8c76-49e7-b15d-8e82c50c80c0\") " pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:38.149373 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.149341 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:38.293775 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.293745 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552851376"] Jul 23 22:38:38.295215 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:38:38.295178 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34f4a295_8c76_49e7_b15d_8e82c50c80c0.slice/crio-9ac0f31c8cd00ddd36681ba0cb412a66c4d61296344f706b015452cf6dba7f64 WatchSource:0}: Error finding container 9ac0f31c8cd00ddd36681ba0cb412a66c4d61296344f706b015452cf6dba7f64: Status 404 returned error can't find the container with id 9ac0f31c8cd00ddd36681ba0cb412a66c4d61296344f706b015452cf6dba7f64 Jul 23 22:38:38.462094 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.462036 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552851376" event={"ID":"34f4a295-8c76-49e7-b15d-8e82c50c80c0","Type":"ContainerStarted","Data":"9ac0f31c8cd00ddd36681ba0cb412a66c4d61296344f706b015452cf6dba7f64"} Jul 23 22:38:38.463369 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.463343 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d55518b4e0f4422fceff6ac7489f1366bf32fa93b07c6ee53f8462f73f3b4af5" Jul 23 22:38:38.463369 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:38.463342 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:38.468943 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:38:38.468915 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7721-139888552700592\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="c98bf967-b888-43a5-aa40-339c6ec01e0e" pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:39.471007 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:39.470972 2579 generic.go:356] "Generic (PLEG): container finished" podID="34f4a295-8c76-49e7-b15d-8e82c50c80c0" containerID="26cfa39512e321f63c06dba05bb1c0071ef9994fee972d12a1794ee42381a7f0" exitCode=0 Jul 23 22:38:39.471411 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:39.471024 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552851376" event={"ID":"34f4a295-8c76-49e7-b15d-8e82c50c80c0","Type":"ContainerDied","Data":"26cfa39512e321f63c06dba05bb1c0071ef9994fee972d12a1794ee42381a7f0"} Jul 23 22:38:39.496233 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:38:39.496135 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7721-139888552700592\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="c98bf967-b888-43a5-aa40-339c6ec01e0e" pod="opendatahub/test-curl-7721-139888552700592" Jul 23 22:38:39.679407 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:39.679373 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c98bf967-b888-43a5-aa40-339c6ec01e0e" path="/var/lib/kubelet/pods/c98bf967-b888-43a5-aa40-339c6ec01e0e/volumes" Jul 23 22:38:40.618459 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.618435 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:40.639175 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.639149 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7721-139888552851376_34f4a295-8c76-49e7-b15d-8e82c50c80c0/test-curl-7721-139888552851376/0.log" Jul 23 22:38:40.664034 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.664005 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7721-139888552851376"] Jul 23 22:38:40.667421 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.667397 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7721-139888552851376"] Jul 23 22:38:40.669502 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.669485 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dc8b\" (UniqueName: \"kubernetes.io/projected/34f4a295-8c76-49e7-b15d-8e82c50c80c0-kube-api-access-4dc8b\") pod \"34f4a295-8c76-49e7-b15d-8e82c50c80c0\" (UID: \"34f4a295-8c76-49e7-b15d-8e82c50c80c0\") " Jul 23 22:38:40.671556 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.671524 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34f4a295-8c76-49e7-b15d-8e82c50c80c0-kube-api-access-4dc8b" (OuterVolumeSpecName: "kube-api-access-4dc8b") pod "34f4a295-8c76-49e7-b15d-8e82c50c80c0" (UID: "34f4a295-8c76-49e7-b15d-8e82c50c80c0"). InnerVolumeSpecName "kube-api-access-4dc8b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:38:40.770824 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.770739 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dc8b\" (UniqueName: \"kubernetes.io/projected/34f4a295-8c76-49e7-b15d-8e82c50c80c0-kube-api-access-4dc8b\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:38:40.833497 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.833452 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:40.833916 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.833902 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34f4a295-8c76-49e7-b15d-8e82c50c80c0" containerName="test-curl-7721-139888552851376" Jul 23 22:38:40.833969 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.833918 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="34f4a295-8c76-49e7-b15d-8e82c50c80c0" containerName="test-curl-7721-139888552851376" Jul 23 22:38:40.834012 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.834002 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="34f4a295-8c76-49e7-b15d-8e82c50c80c0" containerName="test-curl-7721-139888552851376" Jul 23 22:38:40.837037 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.837020 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:40.850046 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.850019 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:40.972298 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:40.972257 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75st\" (UniqueName: \"kubernetes.io/projected/56994d69-2e1f-4f1b-ad72-5a9eba4557e0-kube-api-access-v75st\") pod \"test-curl-7721-139888552785072\" (UID: \"56994d69-2e1f-4f1b-ad72-5a9eba4557e0\") " pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:41.072792 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.072704 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v75st\" (UniqueName: \"kubernetes.io/projected/56994d69-2e1f-4f1b-ad72-5a9eba4557e0-kube-api-access-v75st\") pod \"test-curl-7721-139888552785072\" (UID: \"56994d69-2e1f-4f1b-ad72-5a9eba4557e0\") " pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:41.082243 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.082212 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75st\" (UniqueName: \"kubernetes.io/projected/56994d69-2e1f-4f1b-ad72-5a9eba4557e0-kube-api-access-v75st\") pod \"test-curl-7721-139888552785072\" (UID: \"56994d69-2e1f-4f1b-ad72-5a9eba4557e0\") " pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:41.146906 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.146867 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:41.286287 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.286259 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:41.286980 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:38:41.286954 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56994d69_2e1f_4f1b_ad72_5a9eba4557e0.slice/crio-2334740a7684fbe97e39946d11193cc7e09f82818f4d6967aa37a159b70b316b WatchSource:0}: Error finding container 2334740a7684fbe97e39946d11193cc7e09f82818f4d6967aa37a159b70b316b: Status 404 returned error can't find the container with id 2334740a7684fbe97e39946d11193cc7e09f82818f4d6967aa37a159b70b316b Jul 23 22:38:41.483358 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.483321 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552785072" event={"ID":"56994d69-2e1f-4f1b-ad72-5a9eba4557e0","Type":"ContainerStarted","Data":"2334740a7684fbe97e39946d11193cc7e09f82818f4d6967aa37a159b70b316b"} Jul 23 22:38:41.484476 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.484457 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ac0f31c8cd00ddd36681ba0cb412a66c4d61296344f706b015452cf6dba7f64" Jul 23 22:38:41.484558 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.484478 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:41.495196 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:38:41.495169 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7721-139888552851376\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="34f4a295-8c76-49e7-b15d-8e82c50c80c0" pod="opendatahub/test-curl-7721-139888552851376" Jul 23 22:38:41.679944 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:41.679917 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34f4a295-8c76-49e7-b15d-8e82c50c80c0" path="/var/lib/kubelet/pods/34f4a295-8c76-49e7-b15d-8e82c50c80c0/volumes" Jul 23 22:38:42.491318 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:42.491285 2579 generic.go:356] "Generic (PLEG): container finished" podID="56994d69-2e1f-4f1b-ad72-5a9eba4557e0" containerID="7f3b7ac7b34eb3bf76fc7833f0c6f6dd9f7bf09594ac8276dc7699fac579da94" exitCode=0 Jul 23 22:38:42.491497 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:42.491330 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552785072" event={"ID":"56994d69-2e1f-4f1b-ad72-5a9eba4557e0","Type":"ContainerDied","Data":"7f3b7ac7b34eb3bf76fc7833f0c6f6dd9f7bf09594ac8276dc7699fac579da94"} Jul 23 22:38:43.645474 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.645453 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:43.667218 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.667113 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7721-139888552785072_56994d69-2e1f-4f1b-ad72-5a9eba4557e0/test-curl-7721-139888552785072/0.log" Jul 23 22:38:43.692627 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.692602 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:43.696169 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.696149 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:43.699450 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.699431 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v75st\" (UniqueName: \"kubernetes.io/projected/56994d69-2e1f-4f1b-ad72-5a9eba4557e0-kube-api-access-v75st\") pod \"56994d69-2e1f-4f1b-ad72-5a9eba4557e0\" (UID: \"56994d69-2e1f-4f1b-ad72-5a9eba4557e0\") " Jul 23 22:38:43.701504 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.701483 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56994d69-2e1f-4f1b-ad72-5a9eba4557e0-kube-api-access-v75st" (OuterVolumeSpecName: "kube-api-access-v75st") pod "56994d69-2e1f-4f1b-ad72-5a9eba4557e0" (UID: "56994d69-2e1f-4f1b-ad72-5a9eba4557e0"). InnerVolumeSpecName "kube-api-access-v75st". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:38:43.800229 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.800156 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v75st\" (UniqueName: \"kubernetes.io/projected/56994d69-2e1f-4f1b-ad72-5a9eba4557e0-kube-api-access-v75st\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:38:43.827859 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.827823 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:43.828308 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.828293 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56994d69-2e1f-4f1b-ad72-5a9eba4557e0" containerName="test-curl-7721-139888552785072" Jul 23 22:38:43.828355 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.828313 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="56994d69-2e1f-4f1b-ad72-5a9eba4557e0" containerName="test-curl-7721-139888552785072" Jul 23 22:38:43.828418 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.828408 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="56994d69-2e1f-4f1b-ad72-5a9eba4557e0" containerName="test-curl-7721-139888552785072" Jul 23 22:38:43.831526 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.831510 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:43.842802 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.842776 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:43.901418 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:43.901385 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd7ff\" (UniqueName: \"kubernetes.io/projected/303c7b0a-3055-4cb1-87e8-e2e76a879985-kube-api-access-kd7ff\") pod \"test-curl-7721-139888552785072\" (UID: \"303c7b0a-3055-4cb1-87e8-e2e76a879985\") " pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:44.002395 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.002357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd7ff\" (UniqueName: \"kubernetes.io/projected/303c7b0a-3055-4cb1-87e8-e2e76a879985-kube-api-access-kd7ff\") pod \"test-curl-7721-139888552785072\" (UID: \"303c7b0a-3055-4cb1-87e8-e2e76a879985\") " pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:44.011185 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.011156 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd7ff\" (UniqueName: \"kubernetes.io/projected/303c7b0a-3055-4cb1-87e8-e2e76a879985-kube-api-access-kd7ff\") pod \"test-curl-7721-139888552785072\" (UID: \"303c7b0a-3055-4cb1-87e8-e2e76a879985\") " pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:44.141334 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.141302 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:44.278122 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.278098 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:44.280815 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:38:44.280778 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod303c7b0a_3055_4cb1_87e8_e2e76a879985.slice/crio-bda0eaf8abe8a55e2d1e50186dc56d562274487f6528a27c44d8b8795b0e72d1 WatchSource:0}: Error finding container bda0eaf8abe8a55e2d1e50186dc56d562274487f6528a27c44d8b8795b0e72d1: Status 404 returned error can't find the container with id bda0eaf8abe8a55e2d1e50186dc56d562274487f6528a27c44d8b8795b0e72d1 Jul 23 22:38:44.503863 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.503769 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552785072" event={"ID":"303c7b0a-3055-4cb1-87e8-e2e76a879985","Type":"ContainerStarted","Data":"bda0eaf8abe8a55e2d1e50186dc56d562274487f6528a27c44d8b8795b0e72d1"} Jul 23 22:38:44.505011 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.504992 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2334740a7684fbe97e39946d11193cc7e09f82818f4d6967aa37a159b70b316b" Jul 23 22:38:44.505151 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.505033 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:44.516682 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:44.516651 2579 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7721-139888552785072" oldPodUID="56994d69-2e1f-4f1b-ad72-5a9eba4557e0" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" Jul 23 22:38:45.216178 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:45.216141 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:38:45.512666 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:45.512585 2579 generic.go:356] "Generic (PLEG): container finished" podID="303c7b0a-3055-4cb1-87e8-e2e76a879985" containerID="08a34e08f4bc025112350dd0c422e08dba76dfee28501604610380f6d2c32a05" exitCode=0 Jul 23 22:38:45.512666 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:45.512648 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552785072" event={"ID":"303c7b0a-3055-4cb1-87e8-e2e76a879985","Type":"ContainerDied","Data":"08a34e08f4bc025112350dd0c422e08dba76dfee28501604610380f6d2c32a05"} Jul 23 22:38:45.680278 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:45.680248 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56994d69-2e1f-4f1b-ad72-5a9eba4557e0" path="/var/lib/kubelet/pods/56994d69-2e1f-4f1b-ad72-5a9eba4557e0/volumes" Jul 23 22:38:46.662577 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.662552 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:46.683507 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.683479 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7721-139888552785072_303c7b0a-3055-4cb1-87e8-e2e76a879985/test-curl-7721-139888552785072/0.log" Jul 23 22:38:46.707621 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.707587 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:46.709583 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.709561 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7721-139888552785072"] Jul 23 22:38:46.728646 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.728626 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kd7ff\" (UniqueName: \"kubernetes.io/projected/303c7b0a-3055-4cb1-87e8-e2e76a879985-kube-api-access-kd7ff\") pod \"303c7b0a-3055-4cb1-87e8-e2e76a879985\" (UID: \"303c7b0a-3055-4cb1-87e8-e2e76a879985\") " Jul 23 22:38:46.730675 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.730649 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/303c7b0a-3055-4cb1-87e8-e2e76a879985-kube-api-access-kd7ff" (OuterVolumeSpecName: "kube-api-access-kd7ff") pod "303c7b0a-3055-4cb1-87e8-e2e76a879985" (UID: "303c7b0a-3055-4cb1-87e8-e2e76a879985"). InnerVolumeSpecName "kube-api-access-kd7ff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:38:46.829962 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.829888 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kd7ff\" (UniqueName: \"kubernetes.io/projected/303c7b0a-3055-4cb1-87e8-e2e76a879985-kube-api-access-kd7ff\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:38:46.831583 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.831560 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7721-139888552786080"] Jul 23 22:38:46.832020 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.832008 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" containerName="test-curl-7721-139888552785072" Jul 23 22:38:46.832073 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.832022 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" containerName="test-curl-7721-139888552785072" Jul 23 22:38:46.832118 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.832109 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" containerName="test-curl-7721-139888552785072" Jul 23 22:38:46.835171 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.835155 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:46.840393 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.840366 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552786080"] Jul 23 22:38:46.931434 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:46.931399 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lrsv\" (UniqueName: \"kubernetes.io/projected/a79825a2-3885-4e60-830a-2b445f5d91b6-kube-api-access-9lrsv\") pod \"test-curl-7721-139888552786080\" (UID: \"a79825a2-3885-4e60-830a-2b445f5d91b6\") " pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:47.032950 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.032915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lrsv\" (UniqueName: \"kubernetes.io/projected/a79825a2-3885-4e60-830a-2b445f5d91b6-kube-api-access-9lrsv\") pod \"test-curl-7721-139888552786080\" (UID: \"a79825a2-3885-4e60-830a-2b445f5d91b6\") " pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:47.042494 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.042471 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lrsv\" (UniqueName: \"kubernetes.io/projected/a79825a2-3885-4e60-830a-2b445f5d91b6-kube-api-access-9lrsv\") pod \"test-curl-7721-139888552786080\" (UID: \"a79825a2-3885-4e60-830a-2b445f5d91b6\") " pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:47.147109 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.147077 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:47.283864 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.283841 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7721-139888552786080"] Jul 23 22:38:47.285778 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:38:47.285752 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda79825a2_3885_4e60_830a_2b445f5d91b6.slice/crio-0d867e9fb4d762dcec3036780cbaa483482ef2e534187300c98c5d5be50feedf WatchSource:0}: Error finding container 0d867e9fb4d762dcec3036780cbaa483482ef2e534187300c98c5d5be50feedf: Status 404 returned error can't find the container with id 0d867e9fb4d762dcec3036780cbaa483482ef2e534187300c98c5d5be50feedf Jul 23 22:38:47.526362 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.526281 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:47.526362 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.526290 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bda0eaf8abe8a55e2d1e50186dc56d562274487f6528a27c44d8b8795b0e72d1" Jul 23 22:38:47.527613 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.527578 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552786080" event={"ID":"a79825a2-3885-4e60-830a-2b445f5d91b6","Type":"ContainerStarted","Data":"0d867e9fb4d762dcec3036780cbaa483482ef2e534187300c98c5d5be50feedf"} Jul 23 22:38:47.537694 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:38:47.537670 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7721-139888552785072\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:47.678107 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:38:47.678038 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7721-139888552785072\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" pod="opendatahub/test-curl-7721-139888552785072" Jul 23 22:38:47.679984 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:47.679962 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="303c7b0a-3055-4cb1-87e8-e2e76a879985" path="/var/lib/kubelet/pods/303c7b0a-3055-4cb1-87e8-e2e76a879985/volumes" Jul 23 22:38:48.533696 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:48.533665 2579 generic.go:356] "Generic (PLEG): container finished" podID="a79825a2-3885-4e60-830a-2b445f5d91b6" containerID="e45cd6947321bc173539ca6b1bf782d56e57c1817e9ac3cc14a6de88b02d3d1d" exitCode=0 Jul 23 22:38:48.533873 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:48.533713 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7721-139888552786080" event={"ID":"a79825a2-3885-4e60-830a-2b445f5d91b6","Type":"ContainerDied","Data":"e45cd6947321bc173539ca6b1bf782d56e57c1817e9ac3cc14a6de88b02d3d1d"} Jul 23 22:38:49.688635 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.688610 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:49.710842 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.710817 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7721-139888552786080_a79825a2-3885-4e60-830a-2b445f5d91b6/test-curl-7721-139888552786080/0.log" Jul 23 22:38:49.736625 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.736598 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7721-139888552786080"] Jul 23 22:38:49.739059 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.739030 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7721-139888552786080"] Jul 23 22:38:49.758321 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.758295 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lrsv\" (UniqueName: \"kubernetes.io/projected/a79825a2-3885-4e60-830a-2b445f5d91b6-kube-api-access-9lrsv\") pod \"a79825a2-3885-4e60-830a-2b445f5d91b6\" (UID: \"a79825a2-3885-4e60-830a-2b445f5d91b6\") " Jul 23 22:38:49.760485 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.760455 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a79825a2-3885-4e60-830a-2b445f5d91b6-kube-api-access-9lrsv" (OuterVolumeSpecName: "kube-api-access-9lrsv") pod "a79825a2-3885-4e60-830a-2b445f5d91b6" (UID: "a79825a2-3885-4e60-830a-2b445f5d91b6"). InnerVolumeSpecName "kube-api-access-9lrsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:38:49.859507 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:49.859466 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lrsv\" (UniqueName: \"kubernetes.io/projected/a79825a2-3885-4e60-830a-2b445f5d91b6-kube-api-access-9lrsv\") on node \"ip-10-0-139-32.ec2.internal\" DevicePath \"\"" Jul 23 22:38:50.546752 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:50.546724 2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d867e9fb4d762dcec3036780cbaa483482ef2e534187300c98c5d5be50feedf" Jul 23 22:38:50.546752 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:50.546726 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:50.556407 ip-10-0-139-32 kubenswrapper[2579]: E0723 22:38:50.556377 2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7721-139888552786080\" is forbidden: User \"system:node:ip-10-0-139-32.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-32.ec2.internal' and this object" podUID="a79825a2-3885-4e60-830a-2b445f5d91b6" pod="opendatahub/test-curl-7721-139888552786080" Jul 23 22:38:51.680382 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:38:51.680349 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a79825a2-3885-4e60-830a-2b445f5d91b6" path="/var/lib/kubelet/pods/a79825a2-3885-4e60-830a-2b445f5d91b6/volumes" Jul 23 22:39:06.315296 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:06.315251 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:39:28.618243 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:28.618204 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nlhrb"] Jul 23 22:39:37.655492 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:37.655456 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65696fb76-kbrw9_aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f/authorino/0.log" Jul 23 22:39:43.321717 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:43.321686 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qll9x_342c3d0c-6c49-4af9-9237-d37885218558/manager/0.log" Jul 23 22:39:43.442561 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:43.442530 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wkl4v_589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3/manager/0.log" Jul 23 22:39:43.720156 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:43.720129 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z96zp_7160cbc7-767d-41ca-a42e-c1409b3aa6a2/server/0.log" Jul 23 22:39:43.847882 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:43.847851 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wft6m_3df4a877-bece-4904-a5df-d3b2abcdd064/manager/0.log" Jul 23 22:39:43.966606 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:43.966581 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-2snh4_ac36272a-c11d-4741-bd75-978974926cd7/copy-manifests/0.log" Jul 23 22:39:43.972655 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:43.972593 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-2snh4_ac36272a-c11d-4741-bd75-978974926cd7/manager/0.log" Jul 23 22:39:44.843432 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:44.843405 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29747430-knmnl_c6b0eeaa-d2ef-4c78-8777-f127ce757142/cleanup/0.log" Jul 23 22:39:46.308253 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:46.308222 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65696fb76-kbrw9_aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f/authorino/0.log" Jul 23 22:39:46.438092 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:46.438043 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2kz8b_9c317d40-5abc-4c67-94c2-07977241132b/manager/0.log" Jul 23 22:39:46.553946 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:46.553918 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gksqc_cefbb45b-94ee-4d83-8e39-8b01876d871c/manager/0.log" Jul 23 22:39:46.670943 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:46.670912 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nh6zw_e7a87b00-f781-4f74-85ae-2c94bc2165cd/kuadrant-console-plugin/0.log" Jul 23 22:39:46.792370 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:46.792336 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-nhchq_ef78e830-5c2b-4c2a-80e5-1663e61a040e/registry-server/0.log" Jul 23 22:39:46.923445 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:46.923338 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mmqvg_b4e38394-8399-4bd4-92ec-146cfba6cc5c/manager/0.log" Jul 23 22:39:47.041714 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:47.041690 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nlhrb_bc7e1109-a894-4150-8b08-4eeaa13fdf41/limitador/0.log" Jul 23 22:39:47.168539 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:47.168496 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-svb4z_306be4f4-34a1-48cf-bfcd-d512b29fb078/manager/0.log" Jul 23 22:39:47.819924 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:47.819888 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b78d9f867-6bs87_4fc52e39-8b1e-4ea4-81e9-32b3f97920e9/kube-auth-proxy/0.log" Jul 23 22:39:48.444562 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:48.444524 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fb9c4db6-5x4fg_498767e6-8f98-495f-8d92-82b966ffd20c/router/0.log" Jul 23 22:39:48.820903 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:48.820801 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-8f4xb_b6430080-7336-471f-8872-91c55c667725/main/0.log" Jul 23 22:39:49.180001 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:49.179973 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fqw9p4_f5f7ff3c-6bf4-4440-af13-21e7bd457ce2/main/0.log" Jul 23 22:39:58.084752 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:58.084705 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mgtn9_ac4ced1f-d98f-4f63-94bd-8dd118c70c3e/global-pull-secret-syncer/0.log" Jul 23 22:39:58.169923 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:58.169896 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mrck5_f35f40c2-adf4-4b5c-831f-c9b9f0c40b09/konnectivity-agent/0.log" Jul 23 22:39:58.287893 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:39:58.287857 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-32.ec2.internal_634343fa5cba8d6321e15d7ff451094e/haproxy/0.log" Jul 23 22:40:03.063579 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.063478 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-65696fb76-kbrw9_aaf5cea1-3a4a-4d11-bdf6-c6ef02f3660f/authorino/0.log" Jul 23 22:40:03.088279 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.088253 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-2kz8b_9c317d40-5abc-4c67-94c2-07977241132b/manager/0.log" Jul 23 22:40:03.128392 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.128367 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gksqc_cefbb45b-94ee-4d83-8e39-8b01876d871c/manager/0.log" Jul 23 22:40:03.152312 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.152291 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-nh6zw_e7a87b00-f781-4f74-85ae-2c94bc2165cd/kuadrant-console-plugin/0.log" Jul 23 22:40:03.192263 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.192240 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-nhchq_ef78e830-5c2b-4c2a-80e5-1663e61a040e/registry-server/0.log" Jul 23 22:40:03.327123 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.326987 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mmqvg_b4e38394-8399-4bd4-92ec-146cfba6cc5c/manager/0.log" Jul 23 22:40:03.349137 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.349110 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nlhrb_bc7e1109-a894-4150-8b08-4eeaa13fdf41/limitador/0.log" Jul 23 22:40:03.418415 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:03.418384 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-svb4z_306be4f4-34a1-48cf-bfcd-d512b29fb078/manager/0.log" Jul 23 22:40:05.047278 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.047246 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/alertmanager/0.log" Jul 23 22:40:05.070499 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.070470 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/config-reloader/0.log" Jul 23 22:40:05.090937 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.090917 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/kube-rbac-proxy-web/0.log" Jul 23 22:40:05.112199 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.112177 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/kube-rbac-proxy/0.log" Jul 23 22:40:05.134252 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.134230 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/kube-rbac-proxy-metric/0.log" Jul 23 22:40:05.155405 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.155387 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/prom-label-proxy/0.log" Jul 23 22:40:05.176787 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.176771 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13a90b1a-e89d-4c50-87fe-5b08a0018d79/init-config-reloader/0.log" Jul 23 22:40:05.224888 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.224857 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-phk9x_6cd66ba2-7eba-46a5-8f66-c25231d3e0be/cluster-monitoring-operator/0.log" Jul 23 22:40:05.267302 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.267279 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-blgcc_5c237d47-9086-4a90-9d1e-f7f7f025e271/kube-state-metrics/0.log" Jul 23 22:40:05.286334 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.286312 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-blgcc_5c237d47-9086-4a90-9d1e-f7f7f025e271/kube-rbac-proxy-main/0.log" Jul 23 22:40:05.312575 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.312511 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-blgcc_5c237d47-9086-4a90-9d1e-f7f7f025e271/kube-rbac-proxy-self/0.log" Jul 23 22:40:05.424744 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.424714 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-scn6w_8069444e-71d4-4297-9fc1-4d12103d1dee/monitoring-plugin/0.log" Jul 23 22:40:05.576683 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.576610 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fkrbh_e9bec962-6b68-4b9a-855f-369e9d47ba09/node-exporter/0.log" Jul 23 22:40:05.597793 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.597762 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fkrbh_e9bec962-6b68-4b9a-855f-369e9d47ba09/kube-rbac-proxy/0.log" Jul 23 22:40:05.618462 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.618440 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fkrbh_e9bec962-6b68-4b9a-855f-369e9d47ba09/init-textfile/0.log" Jul 23 22:40:05.749767 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.749723 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-hzs7p_1afe055a-60db-4e83-bf38-559fef5044f8/kube-rbac-proxy-main/0.log" Jul 23 22:40:05.776461 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.776430 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-hzs7p_1afe055a-60db-4e83-bf38-559fef5044f8/kube-rbac-proxy-self/0.log" Jul 23 22:40:05.798067 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:05.798033 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-hzs7p_1afe055a-60db-4e83-bf38-559fef5044f8/openshift-state-metrics/0.log" Jul 23 22:40:06.089597 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:06.089572 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-lpt4s_96444638-fc69-42cc-ad26-d99bc1629138/prometheus-operator-admission-webhook/0.log" Jul 23 22:40:06.117319 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:06.117299 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b69d45f7b-lnwsr_f4e909e3-d953-42c9-8d6a-c09c5bdb3060/telemeter-client/0.log" Jul 23 22:40:06.138468 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:06.138441 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b69d45f7b-lnwsr_f4e909e3-d953-42c9-8d6a-c09c5bdb3060/reload/0.log" Jul 23 22:40:06.163510 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:06.163490 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b69d45f7b-lnwsr_f4e909e3-d953-42c9-8d6a-c09c5bdb3060/kube-rbac-proxy/0.log" Jul 23 22:40:08.158332 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.158301 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6"] Jul 23 22:40:08.158832 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.158808 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a79825a2-3885-4e60-830a-2b445f5d91b6" containerName="test-curl-7721-139888552786080" Jul 23 22:40:08.158832 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.158826 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a79825a2-3885-4e60-830a-2b445f5d91b6" containerName="test-curl-7721-139888552786080" Jul 23 22:40:08.159039 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.158940 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a79825a2-3885-4e60-830a-2b445f5d91b6" containerName="test-curl-7721-139888552786080" Jul 23 22:40:08.162264 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.162246 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.164800 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.164781 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvbz2\"/\"kube-root-ca.crt\"" Jul 23 22:40:08.164915 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.164871 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvbz2\"/\"openshift-service-ca.crt\"" Jul 23 22:40:08.165869 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.165854 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fvbz2\"/\"default-dockercfg-fnxfx\"" Jul 23 22:40:08.171039 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.171016 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6"] Jul 23 22:40:08.310513 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.310478 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-sys\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.310679 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.310560 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-lib-modules\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.310679 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.310588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n58q2\" (UniqueName: \"kubernetes.io/projected/5400df88-4436-42ce-9b5f-c096c942b33b-kube-api-access-n58q2\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.310679 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.310611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-proc\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.310791 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.310690 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-podres\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.381589 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.381561 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/1.log" Jul 23 22:40:08.386630 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.386592 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-kmfq9_af5e4163-1b02-4044-83e1-98a758443469/console-operator/2.log" Jul 23 22:40:08.411590 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-podres\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411590 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-sys\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411640 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-lib-modules\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n58q2\" (UniqueName: \"kubernetes.io/projected/5400df88-4436-42ce-9b5f-c096c942b33b-kube-api-access-n58q2\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411678 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-podres\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411697 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-proc\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411720 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-sys\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411756 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-proc\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.411875 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.411852 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5400df88-4436-42ce-9b5f-c096c942b33b-lib-modules\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.420603 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.420584 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n58q2\" (UniqueName: \"kubernetes.io/projected/5400df88-4436-42ce-9b5f-c096c942b33b-kube-api-access-n58q2\") pod \"perf-node-gather-daemonset-m52m6\" (UID: \"5400df88-4436-42ce-9b5f-c096c942b33b\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.473852 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.473826 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:08.625338 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.625316 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6"] Jul 23 22:40:08.627576 ip-10-0-139-32 kubenswrapper[2579]: W0723 22:40:08.627543 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5400df88_4436_42ce_9b5f_c096c942b33b.slice/crio-c919e40a3cea7f7ce0a0ceee1a071c204b06cb0b90aa93e132397b17c110b41c WatchSource:0}: Error finding container c919e40a3cea7f7ce0a0ceee1a071c204b06cb0b90aa93e132397b17c110b41c: Status 404 returned error can't find the container with id c919e40a3cea7f7ce0a0ceee1a071c204b06cb0b90aa93e132397b17c110b41c Jul 23 22:40:08.937823 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.937727 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8595d64bf8-27gtq_e86782f7-017e-4131-b7d1-b7fb1ca4a209/console/0.log" Jul 23 22:40:08.995335 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.995300 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" event={"ID":"5400df88-4436-42ce-9b5f-c096c942b33b","Type":"ContainerStarted","Data":"66e6da16fbd9e0f000320450e6053689950e3680d9583471bb85ff6f33e0db16"} Jul 23 22:40:08.995492 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.995359 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" event={"ID":"5400df88-4436-42ce-9b5f-c096c942b33b","Type":"ContainerStarted","Data":"c919e40a3cea7f7ce0a0ceee1a071c204b06cb0b90aa93e132397b17c110b41c"} Jul 23 22:40:08.995492 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:08.995391 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:09.011768 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:09.011726 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" podStartSLOduration=1.011710107 podStartE2EDuration="1.011710107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:40:09.011214996 +0000 UTC m=+2921.770286011" watchObservedRunningTime="2026-07-23 22:40:09.011710107 +0000 UTC m=+2921.770781117" Jul 23 22:40:09.633668 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:09.633640 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-x4fzk_93579de5-b2dd-4ba2-976f-007a2059487b/volume-data-source-validator/0.log" Jul 23 22:40:10.748911 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:10.748885 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l4gl4_4de109c8-8b50-46f2-b7ae-e4340a11cdcf/dns/0.log" Jul 23 22:40:10.768606 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:10.768579 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l4gl4_4de109c8-8b50-46f2-b7ae-e4340a11cdcf/kube-rbac-proxy/0.log" Jul 23 22:40:10.862427 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:10.862397 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rqrtl_5af88a0e-a538-435c-a540-65e938e7ecf6/dns-node-resolver/0.log" Jul 23 22:40:11.539438 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:11.539411 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w4lz2_7c95a45f-e8ec-402b-97eb-9089ed4c2bce/node-ca/0.log" Jul 23 22:40:12.636063 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:12.636026 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b78d9f867-6bs87_4fc52e39-8b1e-4ea4-81e9-32b3f97920e9/kube-auth-proxy/0.log" Jul 23 22:40:12.824122 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:12.824092 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fb9c4db6-5x4fg_498767e6-8f98-495f-8d92-82b966ffd20c/router/0.log" Jul 23 22:40:13.532329 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:13.532303 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-966bw_baf45753-a888-4811-b238-767035037d2e/serve-healthcheck-canary/0.log" Jul 23 22:40:14.045872 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:14.045842 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-42jjx_59ed4961-d552-4186-8e13-31f800553994/insights-operator/0.log" Jul 23 22:40:14.046642 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:14.046624 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-42jjx_59ed4961-d552-4186-8e13-31f800553994/insights-operator/1.log" Jul 23 22:40:14.069637 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:14.069616 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5fhxg_5eaafa37-b84a-489c-9d61-75852a35147a/kube-rbac-proxy/0.log" Jul 23 22:40:14.090159 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:14.090137 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5fhxg_5eaafa37-b84a-489c-9d61-75852a35147a/exporter/0.log" Jul 23 22:40:14.111729 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:14.111708 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5fhxg_5eaafa37-b84a-489c-9d61-75852a35147a/extractor/0.log" Jul 23 22:40:15.012315 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:15.012288 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-m52m6" Jul 23 22:40:16.562179 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:16.562149 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qll9x_342c3d0c-6c49-4af9-9237-d37885218558/manager/0.log" Jul 23 22:40:16.760503 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:16.760471 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-wkl4v_589d90cb-0c8d-46bc-9c4d-7dbaa12b75e3/manager/0.log" Jul 23 22:40:16.907044 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:16.907018 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-z96zp_7160cbc7-767d-41ca-a42e-c1409b3aa6a2/server/0.log" Jul 23 22:40:16.976922 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:16.976896 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wft6m_3df4a877-bece-4904-a5df-d3b2abcdd064/manager/0.log" Jul 23 22:40:17.002382 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:17.002355 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-2snh4_ac36272a-c11d-4741-bd75-978974926cd7/manager/0.log" Jul 23 22:40:17.027610 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:17.027587 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-2snh4_ac36272a-c11d-4741-bd75-978974926cd7/copy-manifests/0.log" Jul 23 22:40:18.661007 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:18.660974 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-926mm_1e77d809-9469-40ac-ac6d-150d9ec2ef38/openshift-lws-operator/0.log" Jul 23 22:40:19.181046 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:19.181021 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b78d9f867-6bs87_4fc52e39-8b1e-4ea4-81e9-32b3f97920e9/kube-auth-proxy/0.log" Jul 23 22:40:19.300766 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:19.300744 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fb9c4db6-5x4fg_498767e6-8f98-495f-8d92-82b966ffd20c/router/0.log" Jul 23 22:40:20.648280 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:20.648252 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b78d9f867-6bs87_4fc52e39-8b1e-4ea4-81e9-32b3f97920e9/kube-auth-proxy/0.log" Jul 23 22:40:20.777915 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:20.777885 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fb9c4db6-5x4fg_498767e6-8f98-495f-8d92-82b966ffd20c/router/0.log" Jul 23 22:40:24.256601 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:24.256571 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-6h9vg_8c0c96d0-bbea-4969-a466-5c46ab6285f8/kube-storage-version-migrator-operator/1.log" Jul 23 22:40:24.257404 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:24.257387 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-6h9vg_8c0c96d0-bbea-4969-a466-5c46ab6285f8/kube-storage-version-migrator-operator/0.log" Jul 23 22:40:25.553159 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.553125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/kube-multus-additional-cni-plugins/0.log" Jul 23 22:40:25.573583 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.573545 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/egress-router-binary-copy/0.log" Jul 23 22:40:25.594271 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.594249 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/cni-plugins/0.log" Jul 23 22:40:25.614421 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.614386 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/bond-cni-plugin/0.log" Jul 23 22:40:25.635128 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.635101 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/routeoverride-cni/0.log" Jul 23 22:40:25.657179 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.657152 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/whereabouts-cni-bincopy/0.log" Jul 23 22:40:25.679176 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.679151 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rlmqv_4c4f2c9d-6f0f-43a2-8ad9-35dfd93493b1/whereabouts-cni/0.log" Jul 23 22:40:25.785276 ip-10-0-139-32 kubenswrapper[2579]: I0723 22:40:25.785253 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ntkwx_11144f63-f9a8-4fcc-a309-56aee07f6423/kube-multus/0.log"