Jul 22 18:39:51.352098 ip-10-0-137-148 systemd[1]: Starting Kubernetes Kubelet... Jul 22 18:39:51.801389 ip-10-0-137-148 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:39:51.801389 ip-10-0-137-148 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 18:39:51.801389 ip-10-0-137-148 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:39:51.801389 ip-10-0-137-148 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 18:39:51.801389 ip-10-0-137-148 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 18:39:51.803376 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.803289 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 18:39:51.810189 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810163 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:39:51.810189 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810185 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:39:51.810189 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810190 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:39:51.810189 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810193 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:39:51.810189 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810196 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:39:51.810189 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810199 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810203 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810207 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810210 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810213 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810216 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810219 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810222 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810225 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810227 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810230 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810233 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810236 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810239 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810241 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810244 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810247 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810250 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810252 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810255 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:39:51.810412 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810257 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810260 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810262 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810265 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810268 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810270 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810273 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810275 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810278 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810281 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810283 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810286 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810288 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810291 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810295 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810298 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810300 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810304 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810306 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810309 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:39:51.810901 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810311 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810314 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810317 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810319 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810322 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810326 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810328 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810331 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810334 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810336 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810339 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810341 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810344 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810346 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810349 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810352 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810354 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810356 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810360 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810364 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:39:51.811369 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810367 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810370 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810372 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810375 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810377 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810380 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810385 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810388 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810390 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810393 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810396 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810400 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810402 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810405 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810407 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810412 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810417 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810420 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810423 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:39:51.811864 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810426 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810429 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810432 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810434 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810437 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810440 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810442 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810445 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810447 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810450 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810452 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810455 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810458 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810461 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.810463 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811190 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811196 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811200 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811202 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:39:51.812327 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811205 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811208 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811211 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811214 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811216 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811219 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811222 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811225 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811228 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811230 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811233 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811236 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811238 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811242 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811245 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811247 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811250 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811252 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811255 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811258 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:39:51.812782 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811260 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811263 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811265 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811268 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811271 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811273 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811276 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811278 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811281 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811284 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811287 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811290 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811292 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811295 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811298 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811300 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811304 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811308 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811311 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:39:51.813284 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811313 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811317 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811319 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811321 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811324 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811326 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811329 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811332 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811334 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811337 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811340 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811343 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811345 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811349 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811353 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811357 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811360 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811362 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811365 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:39:51.813746 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811369 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811372 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811375 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811378 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811381 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811384 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811387 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811390 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811393 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811396 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811399 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811402 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811404 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811407 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811410 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811414 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811416 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811419 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811423 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811425 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:39:51.814204 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811428 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811431 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811433 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811436 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811439 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811441 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811443 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811446 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811448 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811451 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811454 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811457 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811460 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811463 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811465 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811468 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811470 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811885 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811891 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811895 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:39:51.814683 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811898 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811900 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811903 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811906 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811908 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811911 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811913 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811916 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811919 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811921 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811924 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811927 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811929 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811932 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811935 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811938 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811940 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811943 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811945 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811947 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:39:51.815245 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811951 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811954 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811956 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811959 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811961 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811964 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811968 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811972 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811975 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811978 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811980 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811983 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811985 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811990 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811993 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811996 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.811999 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812002 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812004 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:39:51.815722 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812007 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812009 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812012 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812014 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812017 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812019 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812021 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812025 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812027 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812030 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812032 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812035 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812037 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812040 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812044 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812046 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812049 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812051 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812054 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812057 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:39:51.816206 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812059 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812061 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812064 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812066 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812069 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812072 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812074 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812077 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812079 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812082 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812084 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812086 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812089 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812092 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812096 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812098 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812101 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812103 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812105 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812108 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:39:51.816714 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812116 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812119 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812122 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812124 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812127 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812130 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812133 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812135 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812138 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812140 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812143 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812145 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812148 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812150 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812153 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812155 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812159 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812224 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812231 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812238 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 18:39:51.817216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812242 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812246 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812249 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812253 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812263 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812267 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812270 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812273 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812276 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812280 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812282 2566 flags.go:64] FLAG: --cgroup-root="" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812285 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812288 2566 flags.go:64] FLAG: --client-ca-file="" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812297 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812301 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812312 2566 flags.go:64] FLAG: --cluster-domain="" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812315 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812321 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812324 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812328 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812333 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812336 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812339 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 18:39:51.817714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812343 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812346 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812349 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812352 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812355 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812358 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812362 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812365 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812368 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812371 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812374 2566 flags.go:64] FLAG: --enable-server="true" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812377 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812382 2566 flags.go:64] FLAG: --event-burst="100" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812385 2566 flags.go:64] FLAG: --event-qps="50" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812388 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812391 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812394 2566 flags.go:64] FLAG: --eviction-hard="" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812398 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812401 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812404 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812407 2566 flags.go:64] FLAG: --eviction-soft="" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812410 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812413 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812417 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812421 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 18:39:51.818308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812424 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812428 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812431 2566 flags.go:64] FLAG: --feature-gates="" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812435 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812438 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812441 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812444 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812447 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812451 2566 flags.go:64] FLAG: --help="false" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812454 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-137-148.ec2.internal" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812457 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812460 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812463 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812466 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812470 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812473 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812476 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812479 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812481 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812484 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812487 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812490 2566 flags.go:64] FLAG: --kube-reserved="" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812493 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812496 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 18:39:51.818936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812499 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812502 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812505 2566 flags.go:64] FLAG: --lock-file="" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812507 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812510 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812513 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812519 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812524 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812527 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812531 2566 flags.go:64] FLAG: --logging-format="text" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812533 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812537 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812540 2566 flags.go:64] FLAG: --manifest-url="" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812543 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812547 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812550 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812555 2566 flags.go:64] FLAG: --max-pods="110" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812557 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812561 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812564 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812567 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812570 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812572 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812575 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812582 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 18:39:51.819552 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812585 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812588 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812592 2566 flags.go:64] FLAG: --pod-cidr="" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812595 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812600 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812602 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812605 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812608 2566 flags.go:64] FLAG: --port="10250" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812612 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812614 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0be2b5d25dba63e5d" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812618 2566 flags.go:64] FLAG: --qos-reserved="" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812621 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812624 2566 flags.go:64] FLAG: --register-node="true" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812626 2566 flags.go:64] FLAG: --register-with-taints="" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812632 2566 flags.go:64] FLAG: --registry-burst="10" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812636 2566 flags.go:64] FLAG: --registry-qps="5" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812639 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812643 2566 flags.go:64] FLAG: --reserved-memory="" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812647 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812650 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812653 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812656 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812658 2566 flags.go:64] FLAG: --runonce="false" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812661 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812664 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 18:39:51.820178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812667 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812670 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812673 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812676 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812679 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812682 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812685 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812688 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812691 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812693 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812696 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812699 2566 flags.go:64] FLAG: --system-cgroups="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812702 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812707 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812710 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812712 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812717 2566 flags.go:64] FLAG: --tls-min-version="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812720 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812722 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812726 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812728 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812732 2566 flags.go:64] FLAG: --v="2" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812736 2566 flags.go:64] FLAG: --version="false" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812742 2566 flags.go:64] FLAG: --vmodule="" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812747 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 18:39:51.820790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.812750 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812888 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812892 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812896 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812899 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812901 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812904 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812907 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812910 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812912 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812915 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812919 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812922 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812925 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812928 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812930 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812933 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812935 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812939 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:39:51.821416 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812943 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812947 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812950 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812953 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812956 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812960 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812963 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812965 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812969 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812973 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812975 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812978 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812983 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812985 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812988 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812991 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812993 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812995 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.812998 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813001 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:39:51.821923 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813003 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813006 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813008 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813011 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813014 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813016 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813018 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813021 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813023 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813026 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813029 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813031 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813034 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813036 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813039 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813041 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813044 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813046 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813048 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813051 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:39:51.822407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813053 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813057 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813060 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813062 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813066 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813073 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813075 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813078 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813081 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813083 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813086 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813088 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813091 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813094 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813096 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813099 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813102 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813104 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813107 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:39:51.822933 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813109 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813112 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813114 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813117 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813119 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813122 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813124 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813127 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813130 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813132 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813135 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813137 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813140 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813143 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813147 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813150 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813152 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813157 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813160 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813163 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:39:51.823381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813166 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.813168 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.813834 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.820619 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.820635 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.820654 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.820660 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820729 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820734 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820738 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820741 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820744 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820747 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820749 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:39:51.823854 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820752 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820755 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820759 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820763 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820776 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820779 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820782 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820785 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820788 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820791 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820794 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820797 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820799 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820802 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820805 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820807 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820810 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820826 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820829 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:39:51.824208 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820832 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820834 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820848 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820851 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820854 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820857 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820860 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820862 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820865 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820868 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820871 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820873 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820876 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820879 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820882 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820886 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820888 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820891 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820894 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:39:51.824658 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820896 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820899 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820901 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820903 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820906 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820908 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820911 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820913 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820916 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820918 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820921 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820924 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820927 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820930 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820933 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820936 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820939 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820942 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820945 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820948 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:39:51.825209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820950 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820953 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820956 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820958 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820961 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820963 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820965 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820968 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820970 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820973 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820975 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820978 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820981 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820983 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820986 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820989 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820991 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820993 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820996 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.820998 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:39:51.825686 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821001 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821003 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821008 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821011 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821014 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821018 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821021 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821024 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821026 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821029 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821032 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821034 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821037 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821039 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:39:51.826175 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.821044 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821156 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821160 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821163 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821166 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821169 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821171 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821174 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821176 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821179 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821182 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821185 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821189 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821192 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821194 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821197 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821200 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821203 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821205 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821208 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 18:39:51.826538 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821211 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821213 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821216 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821219 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821222 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821224 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821227 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821229 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821232 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821234 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821237 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821239 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821242 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821244 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821247 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821249 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821252 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821254 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821256 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821259 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 18:39:51.827149 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821261 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821264 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821267 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821269 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821272 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821274 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821277 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821279 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821282 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821285 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821287 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821291 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821295 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821298 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821300 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821303 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821306 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821309 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821311 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821314 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 18:39:51.827617 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821316 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821319 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821321 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821324 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821326 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821329 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821331 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821333 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821336 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821338 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821341 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821343 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821346 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821348 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821350 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821353 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821355 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821358 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821361 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 18:39:51.828107 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821364 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821367 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821370 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821372 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821375 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821378 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821381 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821384 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821386 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821389 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821391 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821394 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821397 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821399 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821402 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821404 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821407 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821410 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821412 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821415 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 18:39:51.828554 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:51.821417 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.821422 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.821520 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.824666 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.826261 2566 server.go:1019] "Starting client certificate rotation" Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.826356 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.826389 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 18:39:51.829078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.829013 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 18:39:51.831946 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.831772 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 18:39:51.846836 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.846799 2566 log.go:25] "Validated CRI v1 runtime API" Jul 22 18:39:51.849151 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.849136 2566 log.go:25] "Validated CRI v1 image API" Jul 22 18:39:51.849974 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.849951 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 18:39:51.852348 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.852325 2566 fs.go:135] Filesystem UUIDs: map[224554ca-7f10-401f-909e-0a9e43a79de7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9ecec366-9145-433a-97d7-4fdc2f07d145:/dev/nvme0n1p4] Jul 22 18:39:51.852419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.852346 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 18:39:51.855245 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.855227 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 18:39:51.858618 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.858510 2566 manager.go:216] Machine: {Timestamp:2026-07-22 18:39:51.85705724 +0000 UTC m=+0.374868145 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101485 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec228d0353fb373f00114f22cf7507dd SystemUUID:ec228d03-53fb-373f-0011-4f22cf7507dd BootID:ece1fd51-8ac2-456a-a3ec-01e9459bf9b1 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:53:fa:c4:49:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:53:fa:c4:49:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:c2:8a:bd:de:07 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 18:39:51.858618 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.858613 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 18:39:51.858728 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.858689 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 18:39:51.862480 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.862458 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 18:39:51.862622 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.862483 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 18:39:51.863384 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.863374 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 18:39:51.863418 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.863387 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 18:39:51.863418 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.863394 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 18:39:51.863418 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.863411 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 18:39:51.865166 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.865156 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 18:39:51.869683 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869672 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 22 18:39:51.869803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869793 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 18:39:51.869858 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869835 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 22 18:39:51.869858 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869856 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 18:39:51.869910 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869880 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 18:39:51.869910 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869892 2566 kubelet.go:398] "Adding apiserver pod source" Jul 22 18:39:51.869910 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.869902 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 18:39:51.870257 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.870246 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 18:39:51.870304 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.870264 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 18:39:51.872494 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.872474 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 18:39:51.875149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.875136 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 18:39:51.875198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.875166 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877736 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877760 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877769 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877777 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877790 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877810 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877841 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877852 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877862 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877871 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877883 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877892 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 18:39:51.877960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.877908 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmzsh" Jul 22 18:39:51.878838 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.878800 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 18:39:51.878838 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.878837 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 18:39:51.883458 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.879667 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 18:39:51.883458 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.879678 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 18:39:51.883616 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.883600 2566 server.go:1268] "Started kubelet" Jul 22 18:39:51.883691 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.883673 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 18:39:51.883735 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.883703 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rmzsh" Jul 22 18:39:51.883793 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.883710 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 18:39:51.883907 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.883715 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 18:39:51.883907 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.883863 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 18:39:51.884547 ip-10-0-137-148 systemd[1]: Started Kubernetes Kubelet. Jul 22 18:39:51.885193 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.885179 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 22 18:39:51.887666 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.887651 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 18:39:51.889759 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.889738 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 18:39:51.896119 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.896100 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 18:39:51.896119 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.896113 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 18:39:51.896282 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.896197 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:51.896921 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.896895 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 18:39:51.896995 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.896897 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 18:39:51.896995 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.896937 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 18:39:51.897080 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897027 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 22 18:39:51.897080 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897033 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 22 18:39:51.897171 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897117 2566 factory.go:55] Registering systemd factory Jul 22 18:39:51.897171 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897132 2566 factory.go:223] Registration of the systemd container factory successfully Jul 22 18:39:51.897335 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897321 2566 factory.go:153] Registering CRI-O factory Jul 22 18:39:51.897384 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897338 2566 factory.go:223] Registration of the crio container factory successfully Jul 22 18:39:51.897415 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897386 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 18:39:51.897415 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897407 2566 factory.go:103] Registering Raw factory Jul 22 18:39:51.897481 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.897422 2566 manager.go:1196] Started watching for new ooms in manager Jul 22 18:39:51.898480 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.898466 2566 manager.go:349] Starting recovery of all containers Jul 22 18:39:51.899843 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.899804 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:39:51.904490 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.904467 2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-148.ec2.internal\" not found" node="ip-10-0-137-148.ec2.internal" Jul 22 18:39:51.908440 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.908415 2566 manager.go:354] Recovery completed Jul 22 18:39:51.910947 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.910566 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 22 18:39:51.914489 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.914474 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:51.916396 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.916381 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:51.916453 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.916409 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:51.916453 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.916423 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:51.917098 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917084 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 18:39:51.917098 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917097 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 18:39:51.917235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917107 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:51.917235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917116 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 22 18:39:51.917780 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917767 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:51.917862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917796 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:51.917862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.917811 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:51.919154 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.919141 2566 policy_none.go:49] "None policy: Start" Jul 22 18:39:51.919210 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.919158 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 18:39:51.919210 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.919170 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 18:39:51.921100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.921089 2566 policy_none.go:47] "Start" Jul 22 18:39:51.965469 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965448 2566 manager.go:337] "Starting Device Plugin manager" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.965506 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965522 2566 server.go:87] "Starting device plugin registration server" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965765 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965780 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965880 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965980 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:51.965990 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.966574 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 18:39:51.982030 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:51.966615 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.036117 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.036064 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 18:39:52.037208 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.037189 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 18:39:52.037299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.037238 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 18:39:52.037299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.037273 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 18:39:52.037390 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.037315 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 18:39:52.039915 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.039897 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:39:52.066909 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.066847 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:52.067740 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.067722 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:52.067850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.067753 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:52.067850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.067764 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:52.067850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.067787 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.077862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.077845 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.077918 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.077866 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-148.ec2.internal\": node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.091318 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.091295 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.138219 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.138180 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal"] Jul 22 18:39:52.138332 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.138275 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:52.139873 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.139857 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:52.139972 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.139886 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:52.139972 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.139895 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:52.142133 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.142121 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:52.142258 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.142242 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.142301 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.142275 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:52.143014 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.142997 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:52.143082 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.143023 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:52.143082 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.143033 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:52.143082 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.143006 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:52.143167 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.143090 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:52.143167 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.143098 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:52.145181 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.145165 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.145231 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.145195 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 18:39:52.146196 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.146172 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Jul 22 18:39:52.146196 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.146197 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 18:39:52.146382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.146209 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Jul 22 18:39:52.175436 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.175414 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-148.ec2.internal\" not found" node="ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.179787 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.179772 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-148.ec2.internal\" not found" node="ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.191559 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.191537 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.198916 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.198900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab3cd5bbbb772491f25bac31481457ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"ab3cd5bbbb772491f25bac31481457ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.199003 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.198931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab3cd5bbbb772491f25bac31481457ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"ab3cd5bbbb772491f25bac31481457ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.199003 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.198956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0069f85ef90aeb8dad9e048240ec9720-config\") pod \"kube-apiserver-proxy-ip-10-0-137-148.ec2.internal\" (UID: \"0069f85ef90aeb8dad9e048240ec9720\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.292225 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.292186 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.299523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.299496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab3cd5bbbb772491f25bac31481457ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"ab3cd5bbbb772491f25bac31481457ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.299693 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.299531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0069f85ef90aeb8dad9e048240ec9720-config\") pod \"kube-apiserver-proxy-ip-10-0-137-148.ec2.internal\" (UID: \"0069f85ef90aeb8dad9e048240ec9720\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.299693 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.299560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab3cd5bbbb772491f25bac31481457ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"ab3cd5bbbb772491f25bac31481457ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.299693 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.299590 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab3cd5bbbb772491f25bac31481457ca-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"ab3cd5bbbb772491f25bac31481457ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.299693 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.299600 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0069f85ef90aeb8dad9e048240ec9720-config\") pod \"kube-apiserver-proxy-ip-10-0-137-148.ec2.internal\" (UID: \"0069f85ef90aeb8dad9e048240ec9720\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.299693 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.299594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ab3cd5bbbb772491f25bac31481457ca-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"ab3cd5bbbb772491f25bac31481457ca\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.392952 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.392925 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.478451 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.478414 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.482141 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.482119 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Jul 22 18:39:52.494188 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.494020 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.594861 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.594827 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.695350 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.695295 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.795836 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.795780 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.825205 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.825186 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 18:39:52.825637 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.825323 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 18:39:52.825637 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.825383 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 18:39:52.890506 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.890477 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 18:34:52 +0000 UTC" deadline="2028-02-29 00:04:17.116429619 +0000 UTC" Jul 22 18:39:52.890506 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.890504 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14069h24m24.225928995s" Jul 22 18:39:52.895923 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:52.895901 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Jul 22 18:39:52.936244 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.936221 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:39:52.971936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.971875 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:39:52.997347 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:52.997322 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Jul 22 18:39:53.008166 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.008144 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 18:39:53.009241 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.009228 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Jul 22 18:39:53.016442 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.016426 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 18:39:53.351815 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:53.351783 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0069f85ef90aeb8dad9e048240ec9720.slice/crio-b02a144b4eb3a411cdfaec21552a330cbb17dbda74c0bd77adf63c8b2f344168 WatchSource:0}: Error finding container b02a144b4eb3a411cdfaec21552a330cbb17dbda74c0bd77adf63c8b2f344168: Status 404 returned error can't find the container with id b02a144b4eb3a411cdfaec21552a330cbb17dbda74c0bd77adf63c8b2f344168 Jul 22 18:39:53.355985 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.355968 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 18:39:53.744608 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.744577 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:39:53.836196 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.836165 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 18:39:53.870939 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.870900 2566 apiserver.go:52] "Watching apiserver" Jul 22 18:39:53.877274 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.877238 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 18:39:53.877804 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.877777 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-fs7tf","openshift-network-diagnostics/network-check-target-gs8sb","kube-system/konnectivity-agent-bt8k8","openshift-cluster-node-tuning-operator/tuned-472fg","openshift-multus/multus-additional-cni-plugins-vrfdj","openshift-multus/network-metrics-daemon-sjfmj","openshift-network-operator/iptables-alerter-x57m2","openshift-ovn-kubernetes/ovnkube-node-5xm2d","kube-system/global-pull-secret-syncer-8qw2t","kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc","openshift-dns/node-resolver-bwrkx","openshift-image-registry/node-ca-8mls5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal"] Jul 22 18:39:53.880721 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.880424 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:53.883325 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.882821 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.883325 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.883105 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5dxfn\"" Jul 22 18:39:53.883510 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.883338 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.883510 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.883444 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 18:39:53.883663 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.883642 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:53.886008 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.885607 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.886008 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.885687 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 18:39:53.886008 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.885835 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9qjfp\"" Jul 22 18:39:53.886008 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.885948 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.886433 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.886306 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:53.888159 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.888134 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 18:39:53.888276 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.888184 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sk2ts\"" Jul 22 18:39:53.888276 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.888214 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 18:39:53.888553 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.888534 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.890659 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.890622 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.890745 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.890662 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.890745 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.890674 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6pdjq\"" Jul 22 18:39:53.892024 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.891136 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.893088 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.892969 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.893390 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.893247 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q86fk\"" Jul 22 18:39:53.893961 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.893661 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 18:39:53.893961 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.893706 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:53.893961 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:53.893783 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:39:53.893961 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.893814 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 18:39:53.894197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.894104 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 18:39:53.894197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.894164 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.896160 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.896141 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:53.896265 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:53.896213 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:39:53.896465 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.896447 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 18:39:53.898908 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.898887 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.901098 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.901077 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rcmcf\"" Jul 22 18:39:53.901210 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.901139 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 18:39:53.902297 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.902278 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:53.904447 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:53.903871 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906028 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-cni-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906076 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-host\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906112 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-system-cni-dir\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-kubelet\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21492e71-6527-4603-bf9c-fec1c205cc81-konnectivity-ca\") pod \"konnectivity-agent-bt8k8\" (UID: \"21492e71-6527-4603-bf9c-fec1c205cc81\") " pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-tuned\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906243 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-os-release\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906277 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6685c\" (UniqueName: \"kubernetes.io/projected/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-kube-api-access-6685c\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-cni-multus\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-multus-certs\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906380 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-modprobe-d\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906399 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysconfig\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-kubelet-config\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:53.906638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6djdx\" (UniqueName: \"kubernetes.io/projected/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-kube-api-access-6djdx\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb989be8-2cc6-4c03-9331-710903d993a9-cni-binary-copy\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-etc-kubernetes\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65c4db89-8088-48e2-9bc3-ab21f0c74fef-host\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-systemd\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906689 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-var-lib-kubelet\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-cnibin\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906882 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-os-release\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.906960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgdnk\" (UniqueName: \"kubernetes.io/projected/eb11e4f2-3a61-4d10-9905-ac1e2c578560-kube-api-access-lgdnk\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907009 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-system-cni-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysctl-d\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-run\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-sys\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907307 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb11e4f2-3a61-4d10-9905-ac1e2c578560-host-slash\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907340 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-conf-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.907499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907371 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65c4db89-8088-48e2-9bc3-ab21f0c74fef-serviceca\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-lib-modules\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cnibin\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-dbus\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-socket-dir-parent\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-k8s-cni-cncf-io\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907632 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21492e71-6527-4603-bf9c-fec1c205cc81-agent-certs\") pod \"konnectivity-agent-bt8k8\" (UID: \"21492e71-6527-4603-bf9c-fec1c205cc81\") " pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f80d638-390b-4bd6-b3d2-34adef38cdae-tmp\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sf5j\" (UniqueName: \"kubernetes.io/projected/3f80d638-390b-4bd6-b3d2-34adef38cdae-kube-api-access-5sf5j\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907737 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907759 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-netns\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-cni-bin\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907823 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb989be8-2cc6-4c03-9331-710903d993a9-multus-daemon-config\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9n9v\" (UniqueName: \"kubernetes.io/projected/65c4db89-8088-48e2-9bc3-ab21f0c74fef-kube-api-access-n9n9v\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907899 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-kubernetes\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.908389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysctl-conf\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:53.909100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-hostroot\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.909100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.907991 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8nb4\" (UniqueName: \"kubernetes.io/projected/bb989be8-2cc6-4c03-9331-710903d993a9-kube-api-access-k8nb4\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:53.909100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.908023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb11e4f2-3a61-4d10-9905-ac1e2c578560-iptables-alerter-script\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:53.909247 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.909231 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 18:39:53.910239 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.909748 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:53.912601 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.912577 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 18:39:53.912942 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.912918 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 18:39:53.913213 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.913190 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:53.913581 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.913564 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qjgq6\"" Jul 22 18:39:53.913736 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.913714 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 18:39:53.913817 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.913763 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.913898 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.913719 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 18:39:53.913898 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.913842 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.915656 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.915631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:53.915789 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.915770 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 18:39:53.915908 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.915889 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.916011 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.915995 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.916071 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.916041 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cvqqv\"" Jul 22 18:39:53.917511 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.917490 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 18:39:53.918622 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.918598 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 18:39:53.918713 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.918669 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4sqbs\"" Jul 22 18:39:53.935457 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.935435 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-688ng" Jul 22 18:39:53.943684 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.943665 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-688ng" Jul 22 18:39:53.998683 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:53.998627 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 18:39:54.008240 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008203 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-k8s-cni-cncf-io\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.008356 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008244 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sf5j\" (UniqueName: \"kubernetes.io/projected/3f80d638-390b-4bd6-b3d2-34adef38cdae-kube-api-access-5sf5j\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.008356 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.008356 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-etc-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.008356 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.008356 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008340 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-k8s-cni-cncf-io\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/422501a3-4f95-45c0-b601-33d2e4410917-hosts-file\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-cni-bin\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-kubernetes\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008503 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-cni-bin\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008532 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-var-lib-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008564 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-kubernetes\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008600 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-cni-netd\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-sys-fs\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.008675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008670 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-os-release\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-hostroot\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-os-release\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008747 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008750 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-hostroot\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-cni-bin\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008806 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-kubelet-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-kubelet\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008905 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-kubelet\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21492e71-6527-4603-bf9c-fec1c205cc81-konnectivity-ca\") pod \"konnectivity-agent-bt8k8\" (UID: \"21492e71-6527-4603-bf9c-fec1c205cc81\") " pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.008956 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6685c\" (UniqueName: \"kubernetes.io/projected/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-kube-api-access-6685c\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009022 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009083 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-env-overrides\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-socket-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-cni-multus\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009167 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.009230 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysconfig\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-var-lib-cni-multus\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-ovnkube-script-lib\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009327 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysconfig\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009390 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-etc-selinux\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009429 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6djdx\" (UniqueName: \"kubernetes.io/projected/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-kube-api-access-6djdx\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65c4db89-8088-48e2-9bc3-ab21f0c74fef-host\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-systemd\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-kubelet\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009520 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/21492e71-6527-4603-bf9c-fec1c205cc81-konnectivity-ca\") pod \"konnectivity-agent-bt8k8\" (UID: \"21492e71-6527-4603-bf9c-fec1c205cc81\") " pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-node-log\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-os-release\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009589 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-systemd\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009631 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65c4db89-8088-48e2-9bc3-ab21f0c74fef-host\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009635 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysctl-d\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009671 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-sys\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009701 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-os-release\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.010198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009721 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-dbus\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.009753 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82xfl\" (UniqueName: \"kubernetes.io/projected/5b4f7771-df8b-4870-a865-260e76638f68-kube-api-access-82xfl\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009774 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-sys\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009789 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysctl-d\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-registration-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.009837 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:39:54.50979887 +0000 UTC m=+3.027609789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-device-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-dbus\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb11e4f2-3a61-4d10-9905-ac1e2c578560-host-slash\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009958 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eb11e4f2-3a61-4d10-9905-ac1e2c578560-host-slash\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.009982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cnibin\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/422501a3-4f95-45c0-b601-33d2e4410917-tmp-dir\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-socket-dir-parent\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010075 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cnibin\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21492e71-6527-4603-bf9c-fec1c205cc81-agent-certs\") pod \"konnectivity-agent-bt8k8\" (UID: \"21492e71-6527-4603-bf9c-fec1c205cc81\") " pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f80d638-390b-4bd6-b3d2-34adef38cdae-tmp\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.011149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-systemd-units\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svgb4\" (UniqueName: \"kubernetes.io/projected/134ad375-cb6c-4baf-a567-8835355c8071-kube-api-access-svgb4\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-netns\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb989be8-2cc6-4c03-9331-710903d993a9-multus-daemon-config\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9n9v\" (UniqueName: \"kubernetes.io/projected/65c4db89-8088-48e2-9bc3-ab21f0c74fef-kube-api-access-n9n9v\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysctl-conf\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8nb4\" (UniqueName: \"kubernetes.io/projected/bb989be8-2cc6-4c03-9331-710903d993a9-kube-api-access-k8nb4\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb11e4f2-3a61-4d10-9905-ac1e2c578560-iptables-alerter-script\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.010786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-cni-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-host\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011024 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-socket-dir-parent\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011259 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-netns\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011489 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-sysctl-conf\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011521 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-system-cni-dir\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-ovn\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011617 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-tuned\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011623 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eb11e4f2-3a61-4d10-9905-ac1e2c578560-iptables-alerter-script\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.012006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011672 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-ovnkube-config\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-multus-certs\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-modprobe-d\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-kubelet-config\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011841 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bb989be8-2cc6-4c03-9331-710903d993a9-multus-daemon-config\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011875 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-run-netns\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-host-run-multus-certs\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011942 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-log-socket\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.011995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b4f7771-df8b-4870-a865-260e76638f68-ovn-node-metrics-cert\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012037 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb989be8-2cc6-4c03-9331-710903d993a9-cni-binary-copy\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012089 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-etc-kubernetes\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012120 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-var-lib-kubelet\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-slash\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xfdl\" (UniqueName: \"kubernetes.io/projected/422501a3-4f95-45c0-b601-33d2e4410917-kube-api-access-8xfdl\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.012790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-cnibin\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgdnk\" (UniqueName: \"kubernetes.io/projected/eb11e4f2-3a61-4d10-9905-ac1e2c578560-kube-api-access-lgdnk\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-system-cni-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-run\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-lib-modules\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-conf-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65c4db89-8088-48e2-9bc3-ab21f0c74fef-serviceca\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012550 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012560 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-system-cni-dir\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012584 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-systemd\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012692 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-modprobe-d\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-host\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.012963 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-cnibin\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.013162 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-etc-kubernetes\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.013259 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.013411 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bb989be8-2cc6-4c03-9331-710903d993a9-cni-binary-copy\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.013538 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-system-cni-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.013678 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.013610 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-run\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.014750 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.013704 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:54.014750 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.013917 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-kubelet-config\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:54.014750 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.014007 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-cni-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.014750 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.014018 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:39:54.513988139 +0000 UTC m=+3.031799047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:54.014750 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.014238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-var-lib-kubelet\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.014750 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.014335 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3f80d638-390b-4bd6-b3d2-34adef38cdae-lib-modules\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.015115 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.014838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.015115 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.014912 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bb989be8-2cc6-4c03-9331-710903d993a9-multus-conf-dir\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.015337 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.015314 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/65c4db89-8088-48e2-9bc3-ab21f0c74fef-serviceca\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.016330 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.016304 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f80d638-390b-4bd6-b3d2-34adef38cdae-tmp\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.016435 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.016420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/21492e71-6527-4603-bf9c-fec1c205cc81-agent-certs\") pod \"konnectivity-agent-bt8k8\" (UID: \"21492e71-6527-4603-bf9c-fec1c205cc81\") " pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:54.016499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.016474 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3f80d638-390b-4bd6-b3d2-34adef38cdae-etc-tuned\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.018586 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.018562 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6685c\" (UniqueName: \"kubernetes.io/projected/ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2-kube-api-access-6685c\") pod \"multus-additional-cni-plugins-vrfdj\" (UID: \"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2\") " pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.020820 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.020776 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6djdx\" (UniqueName: \"kubernetes.io/projected/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-kube-api-access-6djdx\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:54.021457 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.021439 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sf5j\" (UniqueName: \"kubernetes.io/projected/3f80d638-390b-4bd6-b3d2-34adef38cdae-kube-api-access-5sf5j\") pod \"tuned-472fg\" (UID: \"3f80d638-390b-4bd6-b3d2-34adef38cdae\") " pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.025055 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.025034 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgdnk\" (UniqueName: \"kubernetes.io/projected/eb11e4f2-3a61-4d10-9905-ac1e2c578560-kube-api-access-lgdnk\") pod \"iptables-alerter-x57m2\" (UID: \"eb11e4f2-3a61-4d10-9905-ac1e2c578560\") " pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.025282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.025260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8nb4\" (UniqueName: \"kubernetes.io/projected/bb989be8-2cc6-4c03-9331-710903d993a9-kube-api-access-k8nb4\") pod \"multus-fs7tf\" (UID: \"bb989be8-2cc6-4c03-9331-710903d993a9\") " pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.026402 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.026380 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9n9v\" (UniqueName: \"kubernetes.io/projected/65c4db89-8088-48e2-9bc3-ab21f0c74fef-kube-api-access-n9n9v\") pod \"node-ca-8mls5\" (UID: \"65c4db89-8088-48e2-9bc3-ab21f0c74fef\") " pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.042596 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.042550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" event={"ID":"0069f85ef90aeb8dad9e048240ec9720","Type":"ContainerStarted","Data":"b02a144b4eb3a411cdfaec21552a330cbb17dbda74c0bd77adf63c8b2f344168"} Jul 22 18:39:54.113314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-etc-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113329 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113362 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/422501a3-4f95-45c0-b601-33d2e4410917-hosts-file\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.113463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113392 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-var-lib-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-cni-netd\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-sys-fs\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-cni-bin\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113555 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-kubelet-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113586 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-env-overrides\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113611 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-socket-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-ovnkube-script-lib\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-etc-selinux\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-kubelet\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-node-log\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82xfl\" (UniqueName: \"kubernetes.io/projected/5b4f7771-df8b-4870-a865-260e76638f68-kube-api-access-82xfl\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.113803 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-registration-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-device-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113838 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/422501a3-4f95-45c0-b601-33d2e4410917-tmp-dir\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113871 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-systemd-units\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svgb4\" (UniqueName: \"kubernetes.io/projected/134ad375-cb6c-4baf-a567-8835355c8071-kube-api-access-svgb4\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-ovn\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.113970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-ovnkube-config\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-run-netns\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-log-socket\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114065 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b4f7771-df8b-4870-a865-260e76638f68-ovn-node-metrics-cert\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114108 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-slash\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-slash\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114191 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-sys-fs\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-etc-selinux\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114236 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-kubelet\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-node-log\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.114383 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114355 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-systemd-units\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xfdl\" (UniqueName: \"kubernetes.io/projected/422501a3-4f95-45c0-b601-33d2e4410917-kube-api-access-8xfdl\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114403 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114431 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-systemd\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114482 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114510 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-systemd\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114528 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114565 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-etc-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114604 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114618 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/422501a3-4f95-45c0-b601-33d2e4410917-tmp-dir\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114650 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/422501a3-4f95-45c0-b601-33d2e4410917-hosts-file\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114774 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-run-netns\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114859 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-env-overrides\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.114931 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-var-lib-openvswitch\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115062 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-ovnkube-script-lib\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115168 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b4f7771-df8b-4870-a865-260e76638f68-ovnkube-config\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115186 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-cni-netd\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.115272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115220 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-host-cni-bin\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.116294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-registration-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.116294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-run-ovn\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.116294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-kubelet-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.116294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115288 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b4f7771-df8b-4870-a865-260e76638f68-log-socket\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.116294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115302 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-device-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.116294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.115307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/134ad375-cb6c-4baf-a567-8835355c8071-socket-dir\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.117820 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.117795 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b4f7771-df8b-4870-a865-260e76638f68-ovn-node-metrics-cert\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.121019 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.120992 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:39:54.121019 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.121021 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:39:54.121179 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.121035 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:54.121179 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.121123 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:39:54.621103864 +0000 UTC m=+3.138914773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:54.123489 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.123464 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xfdl\" (UniqueName: \"kubernetes.io/projected/422501a3-4f95-45c0-b601-33d2e4410917-kube-api-access-8xfdl\") pod \"node-resolver-bwrkx\" (UID: \"422501a3-4f95-45c0-b601-33d2e4410917\") " pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.123597 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.123503 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82xfl\" (UniqueName: \"kubernetes.io/projected/5b4f7771-df8b-4870-a865-260e76638f68-kube-api-access-82xfl\") pod \"ovnkube-node-5xm2d\" (UID: \"5b4f7771-df8b-4870-a865-260e76638f68\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.123597 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.123521 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svgb4\" (UniqueName: \"kubernetes.io/projected/134ad375-cb6c-4baf-a567-8835355c8071-kube-api-access-svgb4\") pod \"aws-ebs-csi-driver-node-knxgc\" (UID: \"134ad375-cb6c-4baf-a567-8835355c8071\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.196468 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.196434 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-x57m2" Jul 22 18:39:54.205097 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.205066 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8mls5" Jul 22 18:39:54.207696 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.207652 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb11e4f2_3a61_4d10_9905_ac1e2c578560.slice/crio-24f5277e9fbbb3d448f0c5fb559079799802430adf02347ed534734442d814c6 WatchSource:0}: Error finding container 24f5277e9fbbb3d448f0c5fb559079799802430adf02347ed534734442d814c6: Status 404 returned error can't find the container with id 24f5277e9fbbb3d448f0c5fb559079799802430adf02347ed534734442d814c6 Jul 22 18:39:54.215466 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.215437 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c4db89_8088_48e2_9bc3_ab21f0c74fef.slice/crio-26aa974ec2b870c9f4af4e30ade591df0650cde1782768d81662252bd4c61b35 WatchSource:0}: Error finding container 26aa974ec2b870c9f4af4e30ade591df0650cde1782768d81662252bd4c61b35: Status 404 returned error can't find the container with id 26aa974ec2b870c9f4af4e30ade591df0650cde1782768d81662252bd4c61b35 Jul 22 18:39:54.216347 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.216323 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:39:54.222652 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.222633 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-472fg" Jul 22 18:39:54.225767 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.225739 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21492e71_6527_4603_bf9c_fec1c205cc81.slice/crio-41ed434f0ade5b79220f08319fce774f3d8290b8d547e8845bb5711642b9b529 WatchSource:0}: Error finding container 41ed434f0ade5b79220f08319fce774f3d8290b8d547e8845bb5711642b9b529: Status 404 returned error can't find the container with id 41ed434f0ade5b79220f08319fce774f3d8290b8d547e8845bb5711642b9b529 Jul 22 18:39:54.230665 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.230580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" Jul 22 18:39:54.232772 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.232749 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f80d638_390b_4bd6_b3d2_34adef38cdae.slice/crio-8bcbd5d6a0ea5ee13fa2fdc8690cd1d60382c864f4748f4d31ceb4a007f16286 WatchSource:0}: Error finding container 8bcbd5d6a0ea5ee13fa2fdc8690cd1d60382c864f4748f4d31ceb4a007f16286: Status 404 returned error can't find the container with id 8bcbd5d6a0ea5ee13fa2fdc8690cd1d60382c864f4748f4d31ceb4a007f16286 Jul 22 18:39:54.239096 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.238941 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fs7tf" Jul 22 18:39:54.241270 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.241248 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1eed09_b1dc_4e7e_a5ec_52531ad3f7f2.slice/crio-d403c019145dd7333b80bb3a61421dc61bd49d90d0db74c3d43f3a36f530d4be WatchSource:0}: Error finding container d403c019145dd7333b80bb3a61421dc61bd49d90d0db74c3d43f3a36f530d4be: Status 404 returned error can't find the container with id d403c019145dd7333b80bb3a61421dc61bd49d90d0db74c3d43f3a36f530d4be Jul 22 18:39:54.247897 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.247877 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:39:54.248086 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.248068 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb989be8_2cc6_4c03_9331_710903d993a9.slice/crio-bd3ce720a3eb7f8677ea78ca9cb16adcda41dbcb78f539daaff324a9d1d761cc WatchSource:0}: Error finding container bd3ce720a3eb7f8677ea78ca9cb16adcda41dbcb78f539daaff324a9d1d761cc: Status 404 returned error can't find the container with id bd3ce720a3eb7f8677ea78ca9cb16adcda41dbcb78f539daaff324a9d1d761cc Jul 22 18:39:54.256022 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.255998 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" Jul 22 18:39:54.257987 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.257968 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4f7771_df8b_4870_a865_260e76638f68.slice/crio-efd5e7a189879cc952500e52f2af4c0e6bdc9b3712facd8ece8db3ca946077b3 WatchSource:0}: Error finding container efd5e7a189879cc952500e52f2af4c0e6bdc9b3712facd8ece8db3ca946077b3: Status 404 returned error can't find the container with id efd5e7a189879cc952500e52f2af4c0e6bdc9b3712facd8ece8db3ca946077b3 Jul 22 18:39:54.263376 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.263359 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bwrkx" Jul 22 18:39:54.264957 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.264937 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod134ad375_cb6c_4baf_a567_8835355c8071.slice/crio-fc028e6506578e6071d62ceedeaa136a4aa2b939c06819f18fff959c7e40d74c WatchSource:0}: Error finding container fc028e6506578e6071d62ceedeaa136a4aa2b939c06819f18fff959c7e40d74c: Status 404 returned error can't find the container with id fc028e6506578e6071d62ceedeaa136a4aa2b939c06819f18fff959c7e40d74c Jul 22 18:39:54.272199 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.272176 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod422501a3_4f95_45c0_b601_33d2e4410917.slice/crio-701dfaa0e7208a44a9ceebc7e33f20f81f50c4e13ab43b0aaaf0427aa1bffa0c WatchSource:0}: Error finding container 701dfaa0e7208a44a9ceebc7e33f20f81f50c4e13ab43b0aaaf0427aa1bffa0c: Status 404 returned error can't find the container with id 701dfaa0e7208a44a9ceebc7e33f20f81f50c4e13ab43b0aaaf0427aa1bffa0c Jul 22 18:39:54.395381 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:39:54.395352 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3cd5bbbb772491f25bac31481457ca.slice/crio-49106df3ea4a36f7d01f0422d2673780e67741a4311391fafc6f83d0a94939dc WatchSource:0}: Error finding container 49106df3ea4a36f7d01f0422d2673780e67741a4311391fafc6f83d0a94939dc: Status 404 returned error can't find the container with id 49106df3ea4a36f7d01f0422d2673780e67741a4311391fafc6f83d0a94939dc Jul 22 18:39:54.516947 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.516854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:54.516947 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.516931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:54.517154 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.516965 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:54.517154 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.517044 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:54.517154 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.517055 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:39:55.517033592 +0000 UTC m=+4.034844504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:54.517154 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.517093 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:39:55.517080354 +0000 UTC m=+4.034891247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:54.718438 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.718403 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:54.718609 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.718542 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:39:54.718609 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.718563 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:39:54.718609 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.718580 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:54.718739 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:54.718641 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:39:55.718625476 +0000 UTC m=+4.236436370 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:54.947265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.947216 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:34:54 +0000 UTC" deadline="2028-03-19 02:31:45.219229534 +0000 UTC" Jul 22 18:39:54.947265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:54.947257 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14527h51m50.271978299s" Jul 22 18:39:55.037681 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.037650 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:55.037900 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.037770 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:39:55.046362 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.046328 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fs7tf" event={"ID":"bb989be8-2cc6-4c03-9331-710903d993a9","Type":"ContainerStarted","Data":"bd3ce720a3eb7f8677ea78ca9cb16adcda41dbcb78f539daaff324a9d1d761cc"} Jul 22 18:39:55.048493 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.048447 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerStarted","Data":"d403c019145dd7333b80bb3a61421dc61bd49d90d0db74c3d43f3a36f530d4be"} Jul 22 18:39:55.049960 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.049869 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-472fg" event={"ID":"3f80d638-390b-4bd6-b3d2-34adef38cdae","Type":"ContainerStarted","Data":"8bcbd5d6a0ea5ee13fa2fdc8690cd1d60382c864f4748f4d31ceb4a007f16286"} Jul 22 18:39:55.051208 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.051184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bt8k8" event={"ID":"21492e71-6527-4603-bf9c-fec1c205cc81","Type":"ContainerStarted","Data":"41ed434f0ade5b79220f08319fce774f3d8290b8d547e8845bb5711642b9b529"} Jul 22 18:39:55.053066 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.053015 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8mls5" event={"ID":"65c4db89-8088-48e2-9bc3-ab21f0c74fef","Type":"ContainerStarted","Data":"26aa974ec2b870c9f4af4e30ade591df0650cde1782768d81662252bd4c61b35"} Jul 22 18:39:55.054387 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.054330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x57m2" event={"ID":"eb11e4f2-3a61-4d10-9905-ac1e2c578560","Type":"ContainerStarted","Data":"24f5277e9fbbb3d448f0c5fb559079799802430adf02347ed534734442d814c6"} Jul 22 18:39:55.055653 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.055625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" event={"ID":"ab3cd5bbbb772491f25bac31481457ca","Type":"ContainerStarted","Data":"49106df3ea4a36f7d01f0422d2673780e67741a4311391fafc6f83d0a94939dc"} Jul 22 18:39:55.056938 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.056909 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bwrkx" event={"ID":"422501a3-4f95-45c0-b601-33d2e4410917","Type":"ContainerStarted","Data":"701dfaa0e7208a44a9ceebc7e33f20f81f50c4e13ab43b0aaaf0427aa1bffa0c"} Jul 22 18:39:55.058260 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.058227 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" event={"ID":"134ad375-cb6c-4baf-a567-8835355c8071","Type":"ContainerStarted","Data":"fc028e6506578e6071d62ceedeaa136a4aa2b939c06819f18fff959c7e40d74c"} Jul 22 18:39:55.059837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.059786 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"efd5e7a189879cc952500e52f2af4c0e6bdc9b3712facd8ece8db3ca946077b3"} Jul 22 18:39:55.527272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.526422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:55.527272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.526515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:55.527272 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.526659 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:55.527272 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.526725 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:39:57.526705713 +0000 UTC m=+6.044516609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:55.527272 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.527174 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:55.527272 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.527227 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:39:57.527210594 +0000 UTC m=+6.045021496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:55.729271 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.728569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:55.729271 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.728776 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:39:55.729271 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.728797 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:39:55.729271 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.728811 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:55.729271 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:55.728897 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:39:57.728855554 +0000 UTC m=+6.246686342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:55.948264 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.948182 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 18:34:54 +0000 UTC" deadline="2028-03-09 05:31:36.846140505 +0000 UTC" Jul 22 18:39:55.948264 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:55.948260 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14290h51m40.897884941s" Jul 22 18:39:56.048850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:56.048772 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:56.049035 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:56.048772 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:56.049035 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:56.048925 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:39:56.049035 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:56.049010 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:39:56.072589 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:56.071983 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" event={"ID":"0069f85ef90aeb8dad9e048240ec9720","Type":"ContainerStarted","Data":"5a0c6f8a62633faf9acc6df834664fe35962caae921490479dec3e9c8479e143"} Jul 22 18:39:56.085916 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:56.085827 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" podStartSLOduration=3.085805569 podStartE2EDuration="3.085805569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:39:56.085571494 +0000 UTC m=+4.603382412" watchObservedRunningTime="2026-07-22 18:39:56.085805569 +0000 UTC m=+4.603616488" Jul 22 18:39:57.038542 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:57.038505 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:57.039070 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.038703 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:39:57.542344 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:57.542245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:57.542344 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:57.542327 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:57.542537 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.542475 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:57.542575 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.542544 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:40:01.542524851 +0000 UTC m=+10.060335748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:39:57.543027 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.543000 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:57.543143 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.543062 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:01.543044097 +0000 UTC m=+10.060855015 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:39:57.743529 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:57.743489 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:57.743824 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.743804 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:39:57.743928 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.743835 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:39:57.743928 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.743850 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:57.744030 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:57.743934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:01.743913046 +0000 UTC m=+10.261723942 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:39:58.038590 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:58.038289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:39:58.039086 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:58.038755 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:39:58.039086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:58.038834 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:39:58.039086 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:58.038948 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:39:58.087920 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:58.087784 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bt8k8" event={"ID":"21492e71-6527-4603-bf9c-fec1c205cc81","Type":"ContainerStarted","Data":"1b88381d134b6265d3c8b76842ac537f91b0d5ee81ba99990d25a62acacf843e"} Jul 22 18:39:59.037559 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:39:59.037526 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:39:59.037736 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:39:59.037636 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:00.038600 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:00.038530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:00.039149 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:00.038812 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:00.039225 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:00.039161 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:00.039277 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:00.039241 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:00.190881 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:00.190560 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:40:00.191743 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:00.191718 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:40:00.208146 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:00.208089 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bt8k8" podStartSLOduration=5.254548183 podStartE2EDuration="8.208069551s" totalImagesPullingTime="2.953521368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:39:58.104181692 +0000 UTC m=+6.621992610" watchObservedRunningTime="2026-07-22 18:40:00.208069551 +0000 UTC m=+8.725880471" Jul 22 18:40:01.037778 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:01.037746 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:01.037988 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.037897 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:01.577659 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:01.577570 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:01.577659 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:01.577651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:01.578268 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.577793 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:40:01.578268 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.577923 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:09.577904424 +0000 UTC m=+18.095715322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:40:01.578268 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.577798 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:40:01.578268 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.577991 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:40:09.577977344 +0000 UTC m=+18.095788256 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:40:01.779398 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:01.779356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:01.779591 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.779569 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:40:01.779676 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.779602 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:40:01.779676 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.779617 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:40:01.779676 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:01.779669 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:09.779655842 +0000 UTC m=+18.297466753 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:40:02.038307 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:02.038262 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:02.038487 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:02.038403 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:02.038570 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:02.038519 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:02.038651 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:02.038624 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:03.037713 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:03.037671 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:03.038197 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:03.037802 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:04.042166 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:04.042133 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:04.042605 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:04.042268 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:04.042605 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:04.042583 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:04.042706 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:04.042682 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:05.038173 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:05.038135 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:05.038363 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:05.038289 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:06.037718 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:06.037682 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:06.038169 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:06.037811 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:06.038169 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:06.037860 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:06.038169 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:06.037994 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:07.037883 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:07.037856 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:07.038267 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:07.037965 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:08.037836 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:08.037797 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:08.038019 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:08.037797 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:08.038019 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:08.037945 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:08.038444 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:08.038013 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:09.038293 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:09.038254 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:09.038739 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.038376 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:09.646176 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:09.646135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:09.646355 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:09.646245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:09.646355 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.646315 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:40:09.646355 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.646341 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:40:09.646462 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.646388 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:40:25.646369608 +0000 UTC m=+34.164180500 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:40:09.646462 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.646405 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:25.64639716 +0000 UTC m=+34.164208052 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:40:09.848314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:09.848267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:09.848496 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.848453 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:40:09.848496 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.848480 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:40:09.848496 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.848493 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:40:09.848650 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:09.848560 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:25.848540795 +0000 UTC m=+34.366351690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:40:10.037845 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:10.037750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:10.037845 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:10.037765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:10.038053 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:10.037887 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:10.038053 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:10.037950 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:11.037981 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:11.037948 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:11.038438 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:11.038078 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:12.038398 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:12.038361 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:12.038794 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:12.038480 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:12.038794 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:12.038532 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:12.038794 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:12.038613 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:13.038378 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:13.038343 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:13.038567 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:13.038506 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:14.038322 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:14.038289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:14.038513 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:14.038417 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:14.038513 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:14.038477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:14.038963 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:14.038573 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:14.938659 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:14.938624 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:40:14.938864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:14.938849 2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 22 18:40:14.939535 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:14.939501 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bt8k8" Jul 22 18:40:15.037531 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.037511 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:15.037622 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:15.037604 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:15.122583 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.122533 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2" containerID="566469ebc8c34fdaef174d5bd68026b20538ebe2afa698857e177a6c8a162f5d" exitCode=0 Jul 22 18:40:15.123294 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.122575 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerDied","Data":"566469ebc8c34fdaef174d5bd68026b20538ebe2afa698857e177a6c8a162f5d"} Jul 22 18:40:15.124593 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.124564 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-472fg" event={"ID":"3f80d638-390b-4bd6-b3d2-34adef38cdae","Type":"ContainerStarted","Data":"c0d2cde1dbc522b6d06c439c52bdbc645330e409eb997736061c636875ae3725"} Jul 22 18:40:15.127079 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.127055 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8mls5" event={"ID":"65c4db89-8088-48e2-9bc3-ab21f0c74fef","Type":"ContainerStarted","Data":"7abc84a04d1c1057d53be877588295c34f9b3bc89c6562568f038c7d76ce5905"} Jul 22 18:40:15.128918 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.128765 2566 generic.go:356] "Generic (PLEG): container finished" podID="ab3cd5bbbb772491f25bac31481457ca" containerID="9eff11beb9a4213052956ce466d2d1d1681743ad9107323bc92f413ccb3a6810" exitCode=0 Jul 22 18:40:15.128918 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.128854 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" event={"ID":"ab3cd5bbbb772491f25bac31481457ca","Type":"ContainerDied","Data":"9eff11beb9a4213052956ce466d2d1d1681743ad9107323bc92f413ccb3a6810"} Jul 22 18:40:15.130444 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.130425 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bwrkx" event={"ID":"422501a3-4f95-45c0-b601-33d2e4410917","Type":"ContainerStarted","Data":"a0b68b2500726aeefe9f1bd9049a94d957a8beb1228e4d708d357206213c65c9"} Jul 22 18:40:15.132138 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.132107 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" event={"ID":"134ad375-cb6c-4baf-a567-8835355c8071","Type":"ContainerStarted","Data":"b4ade60786ff84358e3e2e69f3a3a3a3a13af298f7d64ceef55c317c1912deb7"} Jul 22 18:40:15.134452 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.134434 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:40:15.134793 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.134772 2566 generic.go:356] "Generic (PLEG): container finished" podID="5b4f7771-df8b-4870-a865-260e76638f68" containerID="83109da349ca0771d266042493acbfcb6b4dddab5e8df862d94ca1f59d5c22e1" exitCode=1 Jul 22 18:40:15.134892 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.134845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"952cc87bf0b390bba90995029ce4d7bfe55085f053b4e677aebb8b8f7963c90c"} Jul 22 18:40:15.134949 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.134921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"cc2fbc1b1753fc4e0663e784c21520e984ff10a987220bbc242f4d692d10d81e"} Jul 22 18:40:15.134994 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.134969 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerDied","Data":"83109da349ca0771d266042493acbfcb6b4dddab5e8df862d94ca1f59d5c22e1"} Jul 22 18:40:15.135045 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.135018 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"dcc8b24b237a1766ad6bf00f5c1b8943acdc6fb1c4d3620f8caa46c2e90b55bc"} Jul 22 18:40:15.136370 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.136346 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fs7tf" event={"ID":"bb989be8-2cc6-4c03-9331-710903d993a9","Type":"ContainerStarted","Data":"3785ced8cdd35136509153492d59ea9f4a43c960ed8c70b6bedce008bbbb4173"} Jul 22 18:40:15.155735 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.155692 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-472fg" podStartSLOduration=2.9610059619999998 podStartE2EDuration="23.155677596s" totalImagesPullingTime="20.194671634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:15.155472437 +0000 UTC m=+23.673283351" watchObservedRunningTime="2026-07-22 18:40:15.155677596 +0000 UTC m=+23.673488509" Jul 22 18:40:15.171971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.171933 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fs7tf" podStartSLOduration=2.957549654 podStartE2EDuration="23.171920516s" totalImagesPullingTime="20.214370862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:15.17134663 +0000 UTC m=+23.689157570" watchObservedRunningTime="2026-07-22 18:40:15.171920516 +0000 UTC m=+23.689731463" Jul 22 18:40:15.198317 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.198278 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bwrkx" podStartSLOduration=3.063101474 podStartE2EDuration="23.198265695s" totalImagesPullingTime="20.135164221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:15.197976536 +0000 UTC m=+23.715787452" watchObservedRunningTime="2026-07-22 18:40:15.198265695 +0000 UTC m=+23.716076608" Jul 22 18:40:15.210727 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.210688 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8mls5" podStartSLOduration=3.019272249 podStartE2EDuration="23.210675046s" totalImagesPullingTime="20.191402797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:15.210146471 +0000 UTC m=+23.727957387" watchObservedRunningTime="2026-07-22 18:40:15.210675046 +0000 UTC m=+23.728485960" Jul 22 18:40:15.876872 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.876813 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 18:40:15.981699 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.981565 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T18:40:15.876852285Z","UUID":"befa8bde-c0de-4440-83f3-a901fd2fc373","Handler":null,"Name":"","Endpoint":""} Jul 22 18:40:15.983319 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.983296 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 18:40:15.983452 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:15.983342 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 18:40:16.038170 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.038141 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:16.038170 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.038166 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:16.038395 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:16.038266 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:16.038457 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:16.038400 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:16.140648 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.140603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-x57m2" event={"ID":"eb11e4f2-3a61-4d10-9905-ac1e2c578560","Type":"ContainerStarted","Data":"2f76b6e528e631c4eae30b53918856416dcf87ad1a3ea3b80131d7b47b76f966"} Jul 22 18:40:16.142973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.142943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" event={"ID":"ab3cd5bbbb772491f25bac31481457ca","Type":"ContainerStarted","Data":"181f1c544a0a20306a198cf6c9c45eda7afdc42a516708089ffb55fa4602d9bf"} Jul 22 18:40:16.145149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.145118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" event={"ID":"134ad375-cb6c-4baf-a567-8835355c8071","Type":"ContainerStarted","Data":"87e911676d6e9cd61dacce19b3e1d40e85802243893b17a290b85b86ef7fcfb9"} Jul 22 18:40:16.148350 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.148333 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:40:16.148811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.148789 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"6b5aa4e6b1094d9c53752ac1311620ab1b171dff1ce6b308b1e1082629b4e11d"} Jul 22 18:40:16.148927 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.148887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"f9efe43d59fabebe64a57a533343e567d2c7a5158b073ffa0125db998425a31f"} Jul 22 18:40:16.153932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.153894 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-x57m2" podStartSLOduration=3.935995569 podStartE2EDuration="24.153881363s" totalImagesPullingTime="20.217885794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:16.153531601 +0000 UTC m=+24.671342517" watchObservedRunningTime="2026-07-22 18:40:16.153881363 +0000 UTC m=+24.671692276" Jul 22 18:40:16.170374 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:16.170335 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" podStartSLOduration=23.170323682 podStartE2EDuration="23.170323682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:16.169925951 +0000 UTC m=+24.687736866" watchObservedRunningTime="2026-07-22 18:40:16.170323682 +0000 UTC m=+24.688134595" Jul 22 18:40:17.037685 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:17.037648 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:17.037942 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:17.037875 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:18.038577 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:18.038541 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:18.039071 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:18.038541 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:18.039071 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:18.038709 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:18.039071 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:18.038744 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:18.156061 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:18.156021 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" event={"ID":"134ad375-cb6c-4baf-a567-8835355c8071","Type":"ContainerStarted","Data":"196f3dc62dc24d25a2b8f6783b0546fbea88597d2e0d1d798bab4424786ca306"} Jul 22 18:40:18.159579 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:18.159558 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:40:18.160053 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:18.160031 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"63ea62098352e2e68828dba7a9e5975a7c61669858960b6a212a8656a6258702"} Jul 22 18:40:18.172574 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:18.172533 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-knxgc" podStartSLOduration=3.624283792 podStartE2EDuration="26.172516805s" totalImagesPullingTime="22.548233013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:18.172130253 +0000 UTC m=+26.689941196" watchObservedRunningTime="2026-07-22 18:40:18.172516805 +0000 UTC m=+26.690327718" Jul 22 18:40:19.038022 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:19.037798 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:19.038189 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:19.038127 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:20.038299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.038112 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:20.039000 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.038197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:20.039000 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:20.038378 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:20.039000 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:20.038497 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:20.168422 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.168395 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:40:20.168768 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.168747 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"e8dbfd6b082442d2f576191314c8e82eaeacef7f7687b3927e4d69c3c1aa3e50"} Jul 22 18:40:20.169014 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.168991 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:40:20.169112 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.169072 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:40:20.169193 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.169177 2566 scope.go:117] "RemoveContainer" containerID="83109da349ca0771d266042493acbfcb6b4dddab5e8df862d94ca1f59d5c22e1" Jul 22 18:40:20.172404 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.172377 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2" containerID="c5cd40ba530c0286ce97bb78e8d4f12a657631d7db17d60e6ffaa3d9e1d223c7" exitCode=0 Jul 22 18:40:20.172492 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.172424 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerDied","Data":"c5cd40ba530c0286ce97bb78e8d4f12a657631d7db17d60e6ffaa3d9e1d223c7"} Jul 22 18:40:20.185732 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:20.185712 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:40:21.038128 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.038098 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:21.038332 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:21.038209 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:21.178701 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.178670 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:40:21.179113 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.179075 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" event={"ID":"5b4f7771-df8b-4870-a865-260e76638f68","Type":"ContainerStarted","Data":"27a5ff8237c079c26e1c4d93bd8314cbc51e5347a5ce85ffb0953c25d730cad5"} Jul 22 18:40:21.179274 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.179259 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:40:21.195429 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.195403 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:40:21.209419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.209372 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" podStartSLOduration=8.897195567 podStartE2EDuration="29.209355783s" totalImagesPullingTime="20.312160216s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:21.208394009 +0000 UTC m=+29.726204949" watchObservedRunningTime="2026-07-22 18:40:21.209355783 +0000 UTC m=+29.727166696" Jul 22 18:40:21.484276 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.484237 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8qw2t"] Jul 22 18:40:21.484446 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.484414 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:21.484527 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:21.484507 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:21.487023 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.487001 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gs8sb"] Jul 22 18:40:21.487123 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.487085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:21.487180 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:21.487150 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:21.498061 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.498037 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sjfmj"] Jul 22 18:40:21.498158 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:21.498116 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:21.498200 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:21.498185 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:22.183368 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:22.183335 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2" containerID="70c8953ed1c531e067d403fd7812a8a75127814e93806367bb590c7129664001" exitCode=0 Jul 22 18:40:22.183915 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:22.183416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerDied","Data":"70c8953ed1c531e067d403fd7812a8a75127814e93806367bb590c7129664001"} Jul 22 18:40:23.037497 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:23.037462 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:23.037733 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:23.037579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:23.037733 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:23.037612 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:23.037733 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:23.037612 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:23.037733 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:23.037699 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:23.037960 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:23.037778 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:24.189847 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:24.189800 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2" containerID="df08a0e7bce38010ff3b16413475c3daae4169cdfc666a05fdcb271d330c5685" exitCode=0 Jul 22 18:40:24.190527 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:24.189851 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerDied","Data":"df08a0e7bce38010ff3b16413475c3daae4169cdfc666a05fdcb271d330c5685"} Jul 22 18:40:25.037733 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:25.037694 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:25.037733 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:25.037719 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:25.037967 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.037879 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:25.037967 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:25.037891 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:25.038071 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.037977 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:25.038120 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.038083 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:25.672997 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:25.672958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:25.673632 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:25.673039 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:25.673632 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.673144 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:40:25.673632 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.673167 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 18:40:25.673632 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.673209 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:57.673190879 +0000 UTC m=+66.191001771 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 18:40:25.673632 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.673224 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret podName:5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a nodeName:}" failed. No retries permitted until 2026-07-22 18:40:57.673217668 +0000 UTC m=+66.191028560 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret") pod "global-pull-secret-syncer-8qw2t" (UID: "5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a") : object "kube-system"/"original-pull-secret" not registered Jul 22 18:40:25.875086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:25.875040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:25.875264 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.875236 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 18:40:25.875318 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.875266 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 18:40:25.875318 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.875281 2566 projected.go:196] Error preparing data for projected volume kube-api-access-q275r for pod openshift-network-diagnostics/network-check-target-gs8sb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:40:25.875394 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:25.875353 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r podName:eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:57.875329367 +0000 UTC m=+66.393140271 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q275r" (UniqueName: "kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r") pod "network-check-target-gs8sb" (UID: "eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 18:40:27.038139 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.037808 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:27.038139 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.037811 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:27.038139 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.037937 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gs8sb" podUID="eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99" Jul 22 18:40:27.038139 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.038009 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8qw2t" podUID="5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a" Jul 22 18:40:27.038139 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.037845 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:27.038609 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.038173 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:40:27.233966 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.233936 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeReady" Jul 22 18:40:27.234130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.234082 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 18:40:27.264947 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.264912 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-785cd8c857-vhdvr"] Jul 22 18:40:27.267387 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.267357 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.269568 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.269544 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 18:40:27.269702 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.269672 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 18:40:27.269702 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.269678 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh2s\"" Jul 22 18:40:27.269835 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.269766 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 18:40:27.274945 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.274926 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 18:40:27.280828 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.280792 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ffc88"] Jul 22 18:40:27.283233 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.283212 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tdkfp"] Jul 22 18:40:27.283351 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.283333 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.284912 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.284890 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-785cd8c857-vhdvr"] Jul 22 18:40:27.285023 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.284988 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.285788 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.285770 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 18:40:27.285975 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.285789 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hbkg4\"" Jul 22 18:40:27.286049 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.286014 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 18:40:27.286925 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.286892 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 18:40:27.287168 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.287150 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 18:40:27.287267 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.287235 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 18:40:27.287603 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.287584 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zqsx4\"" Jul 22 18:40:27.294963 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.294295 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ffc88"] Jul 22 18:40:27.295130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.295110 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tdkfp"] Jul 22 18:40:27.384528 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/965c0244-14f9-4460-9e1a-62c696692420-config-volume\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.384667 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7k9c\" (UniqueName: \"kubernetes.io/projected/71b46f3d-ad01-4915-91d3-03d2e769d731-kube-api-access-c7k9c\") pod \"ingress-canary-tdkfp\" (UID: \"71b46f3d-ad01-4915-91d3-03d2e769d731\") " pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.384667 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.384667 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c73ba3b-7767-41d8-9900-be044340308f-ca-trust-extracted\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.384667 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384655 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-registry-certificates\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.384878 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384677 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-trusted-ca\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.384878 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-installation-pull-secrets\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.384878 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8wlm\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-kube-api-access-z8wlm\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.384878 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384787 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b46f3d-ad01-4915-91d3-03d2e769d731-cert\") pod \"ingress-canary-tdkfp\" (UID: \"71b46f3d-ad01-4915-91d3-03d2e769d731\") " pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.384878 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.385052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384881 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n566b\" (UniqueName: \"kubernetes.io/projected/965c0244-14f9-4460-9e1a-62c696692420-kube-api-access-n566b\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.385052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-image-registry-private-configuration\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.385052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384965 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-bound-sa-token\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.385052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.384994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965c0244-14f9-4460-9e1a-62c696692420-tmp-dir\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.485809 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.485775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.485984 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.485827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n566b\" (UniqueName: \"kubernetes.io/projected/965c0244-14f9-4460-9e1a-62c696692420-kube-api-access-n566b\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.485984 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.485866 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-image-registry-private-configuration\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.485984 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.485895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-bound-sa-token\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.485984 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.485923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965c0244-14f9-4460-9e1a-62c696692420-tmp-dir\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.485984 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.485938 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:27.485984 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.485970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/965c0244-14f9-4460-9e1a-62c696692420-config-volume\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.486005 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:27.985984706 +0000 UTC m=+36.503795597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486031 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7k9c\" (UniqueName: \"kubernetes.io/projected/71b46f3d-ad01-4915-91d3-03d2e769d731-kube-api-access-c7k9c\") pod \"ingress-canary-tdkfp\" (UID: \"71b46f3d-ad01-4915-91d3-03d2e769d731\") " pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c73ba3b-7767-41d8-9900-be044340308f-ca-trust-extracted\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-registry-certificates\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-trusted-ca\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-installation-pull-secrets\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486213 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8wlm\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-kube-api-access-z8wlm\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.486310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b46f3d-ad01-4915-91d3-03d2e769d731-cert\") pod \"ingress-canary-tdkfp\" (UID: \"71b46f3d-ad01-4915-91d3-03d2e769d731\") " pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.486785 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486316 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/965c0244-14f9-4460-9e1a-62c696692420-tmp-dir\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.486785 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.486328 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:27.486785 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.486346 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:27.486785 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.486444 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:27.986426506 +0000 UTC m=+36.504237415 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:27.486785 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/965c0244-14f9-4460-9e1a-62c696692420-config-volume\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.487043 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.486899 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-registry-certificates\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.487325 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.487276 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c73ba3b-7767-41d8-9900-be044340308f-ca-trust-extracted\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.487750 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.487728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-trusted-ca\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.491604 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.491434 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-image-registry-private-configuration\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.492365 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.492339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-installation-pull-secrets\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.495989 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.494751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8wlm\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-kube-api-access-z8wlm\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.496094 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.496067 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-bound-sa-token\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.496349 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.496286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n566b\" (UniqueName: \"kubernetes.io/projected/965c0244-14f9-4460-9e1a-62c696692420-kube-api-access-n566b\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.496576 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.496555 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b46f3d-ad01-4915-91d3-03d2e769d731-cert\") pod \"ingress-canary-tdkfp\" (UID: \"71b46f3d-ad01-4915-91d3-03d2e769d731\") " pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.497164 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.497143 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7k9c\" (UniqueName: \"kubernetes.io/projected/71b46f3d-ad01-4915-91d3-03d2e769d731-kube-api-access-c7k9c\") pod \"ingress-canary-tdkfp\" (UID: \"71b46f3d-ad01-4915-91d3-03d2e769d731\") " pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.601240 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.601114 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tdkfp" Jul 22 18:40:27.991263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.991230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:27.991427 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:27.991300 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:27.991427 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.991400 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:27.991427 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.991404 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:27.991427 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.991419 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:27.991635 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.991466 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:28.991446982 +0000 UTC m=+37.509257899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:27.991635 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:27.991483 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:28.991475022 +0000 UTC m=+37.509285914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:28.413314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:28.413286 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tdkfp"] Jul 22 18:40:28.417471 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:40:28.417443 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b46f3d_ad01_4915_91d3_03d2e769d731.slice/crio-1f76ad78a5aabcd1641eeb18416d9f57f11c11c2d70781e5808b587591b246ce WatchSource:0}: Error finding container 1f76ad78a5aabcd1641eeb18416d9f57f11c11c2d70781e5808b587591b246ce: Status 404 returned error can't find the container with id 1f76ad78a5aabcd1641eeb18416d9f57f11c11c2d70781e5808b587591b246ce Jul 22 18:40:28.999759 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:28.999724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:28.999939 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:28.999788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:28.999939 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:28.999889 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:28.999939 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:28.999901 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:29.000034 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:28.999947 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:30.999933408 +0000 UTC m=+39.517744300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:29.000034 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:28.999889 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:29.000034 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:29.000024 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:31.000009447 +0000 UTC m=+39.517820395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:29.038165 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.038134 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:29.038295 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.038228 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:29.038295 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.038257 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:29.040339 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.040313 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 18:40:29.040475 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.040378 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 18:40:29.041122 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.041107 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-csrvs\"" Jul 22 18:40:29.041197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.041164 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 18:40:29.041260 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.041226 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 18:40:29.041314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.041241 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vpr\"" Jul 22 18:40:29.204561 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.204520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tdkfp" event={"ID":"71b46f3d-ad01-4915-91d3-03d2e769d731","Type":"ContainerStarted","Data":"1f76ad78a5aabcd1641eeb18416d9f57f11c11c2d70781e5808b587591b246ce"} Jul 22 18:40:29.207231 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.207202 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2" containerID="a86c1e768f6b6cdffbb910b3f053f8b4e1b4d3de683918ae54f21e5373613204" exitCode=0 Jul 22 18:40:29.207381 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:29.207245 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerDied","Data":"a86c1e768f6b6cdffbb910b3f053f8b4e1b4d3de683918ae54f21e5373613204"} Jul 22 18:40:30.212833 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:30.212649 2566 generic.go:356] "Generic (PLEG): container finished" podID="ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2" containerID="8fcbe402e8be1258bac10af1e15f40ed71636a3230b37e118d1c33d0f2c7e206" exitCode=0 Jul 22 18:40:30.213172 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:30.212706 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerDied","Data":"8fcbe402e8be1258bac10af1e15f40ed71636a3230b37e118d1c33d0f2c7e206"} Jul 22 18:40:31.015113 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:31.015056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:31.015305 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:31.015133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:31.015305 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:31.015225 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:31.015305 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:31.015234 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:31.015305 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:31.015256 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:31.015305 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:31.015277 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:35.015264058 +0000 UTC m=+43.533074950 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:31.015305 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:31.015309 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:35.015292113 +0000 UTC m=+43.533103022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:31.218082 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:31.218048 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" event={"ID":"ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2","Type":"ContainerStarted","Data":"37a301e288f0d1371b940bc34ff3c8403650b500b0fdc40c88922d3b235d0f39"} Jul 22 18:40:31.239136 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:31.239089 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vrfdj" podStartSLOduration=8.715984746 podStartE2EDuration="39.239073329s" totalImagesPullingTime="30.523088583s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:31.238480065 +0000 UTC m=+39.756290983" watchObservedRunningTime="2026-07-22 18:40:31.239073329 +0000 UTC m=+39.756884242" Jul 22 18:40:35.043482 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:35.043446 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:35.043836 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:35.043514 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:35.043836 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:35.043610 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:35.043836 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:35.043615 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:35.043836 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:35.043690 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:43.043672344 +0000 UTC m=+51.561483238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:35.043836 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:35.043620 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:35.043836 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:35.043730 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:43.043724159 +0000 UTC m=+51.561535051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:36.231454 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:36.231410 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tdkfp" event={"ID":"71b46f3d-ad01-4915-91d3-03d2e769d731","Type":"ContainerStarted","Data":"e3cee7680260a3bdca04f822d3ada10414b724c069bee411cf04e3646594326c"} Jul 22 18:40:36.245379 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:36.245326 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tdkfp" podStartSLOduration=2.416092212 podStartE2EDuration="9.245309689s" totalImagesPullingTime="6.829217477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:40:36.244586445 +0000 UTC m=+44.762397356" watchObservedRunningTime="2026-07-22 18:40:36.245309689 +0000 UTC m=+44.763120604" Jul 22 18:40:43.097779 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:43.097744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:43.098280 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:43.097812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:43.098280 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:43.097917 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:43.098280 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:43.097978 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:40:59.097961514 +0000 UTC m=+67.615772406 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:43.098280 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:43.097927 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:43.098280 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:43.098011 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:43.098280 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:43.098052 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:40:59.098041104 +0000 UTC m=+67.615851996 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:53.195514 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:53.195481 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xm2d" Jul 22 18:40:57.698466 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.698395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:57.698869 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.698496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:40:57.700839 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.700795 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 18:40:57.700936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.700891 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 18:40:57.709509 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:57.709482 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 18:40:57.709619 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:57.709584 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:42:01.709561419 +0000 UTC m=+130.227372332 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : secret "metrics-daemon-secret" not found Jul 22 18:40:57.712874 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.712855 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a-original-pull-secret\") pod \"global-pull-secret-syncer-8qw2t\" (UID: \"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a\") " pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:57.847195 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.847153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8qw2t" Jul 22 18:40:57.899952 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.899921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:57.902444 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.902414 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 18:40:57.912204 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.912059 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 18:40:57.923050 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.923018 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q275r\" (UniqueName: \"kubernetes.io/projected/eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99-kube-api-access-q275r\") pod \"network-check-target-gs8sb\" (UID: \"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99\") " pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:57.964970 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:57.964897 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8qw2t"] Jul 22 18:40:57.968407 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:40:57.968382 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a250b6a_0e9c_4276_9d8e_3ccf85c7dd1a.slice/crio-a5dc1999e6d70216313cac427503e84354b71a8020e0299cba18e027b8b0f3dc WatchSource:0}: Error finding container a5dc1999e6d70216313cac427503e84354b71a8020e0299cba18e027b8b0f3dc: Status 404 returned error can't find the container with id a5dc1999e6d70216313cac427503e84354b71a8020e0299cba18e027b8b0f3dc Jul 22 18:40:58.155560 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:58.155529 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-csrvs\"" Jul 22 18:40:58.164285 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:58.164255 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:40:58.279903 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:58.279873 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gs8sb"] Jul 22 18:40:58.283257 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:40:58.283234 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb9fcb5_6acd_42a8_9bcc_8f4f98346e99.slice/crio-67696bc13936d64544ad3b4ca79aba35f2e3fdddee521508f50a9e6fc21eb67c WatchSource:0}: Error finding container 67696bc13936d64544ad3b4ca79aba35f2e3fdddee521508f50a9e6fc21eb67c: Status 404 returned error can't find the container with id 67696bc13936d64544ad3b4ca79aba35f2e3fdddee521508f50a9e6fc21eb67c Jul 22 18:40:58.285967 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:58.285939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8qw2t" event={"ID":"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a","Type":"ContainerStarted","Data":"a5dc1999e6d70216313cac427503e84354b71a8020e0299cba18e027b8b0f3dc"} Jul 22 18:40:59.109413 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:59.109372 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:40:59.109903 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:59.109458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:40:59.109903 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:59.109516 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:40:59.109903 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:59.109601 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:41:31.10957505 +0000 UTC m=+99.627390893 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:40:59.109903 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:59.109600 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:40:59.109903 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:59.109618 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:40:59.109903 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:40:59.109647 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:41:31.10963756 +0000 UTC m=+99.627448475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:40:59.290568 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:40:59.290524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gs8sb" event={"ID":"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99","Type":"ContainerStarted","Data":"67696bc13936d64544ad3b4ca79aba35f2e3fdddee521508f50a9e6fc21eb67c"} Jul 22 18:41:03.301725 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:03.301690 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8qw2t" event={"ID":"5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a","Type":"ContainerStarted","Data":"a4fe41835cc76991c1d6db094d14654e15490fe720e2e31fc09f7e644555f8ad"} Jul 22 18:41:03.316837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:03.316766 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8qw2t" podStartSLOduration=66.701052475 podStartE2EDuration="1m11.316746394s" totalImagesPullingTime="4.615693919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:41:03.316007793 +0000 UTC m=+71.833818706" watchObservedRunningTime="2026-07-22 18:41:03.316746394 +0000 UTC m=+71.834557309" Jul 22 18:41:05.307320 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:05.307292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gs8sb" event={"ID":"eeb9fcb5-6acd-42a8-9bcc-8f4f98346e99","Type":"ContainerStarted","Data":"884857e471ab7bb4be3e156dd180ded6f8e34ddc1cfd2a995a9b49c822703c40"} Jul 22 18:41:05.307701 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:05.307420 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:41:05.321451 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:05.321403 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gs8sb" podStartSLOduration=66.35669325 podStartE2EDuration="1m13.321356374s" totalImagesPullingTime="6.964663124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:41:05.320724721 +0000 UTC m=+73.838535646" watchObservedRunningTime="2026-07-22 18:41:05.321356374 +0000 UTC m=+73.839167289" Jul 22 18:41:31.125652 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:31.125602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:41:31.126168 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:31.125680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:41:31.126168 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:41:31.125765 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:41:31.126168 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:41:31.125781 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:41:31.126168 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:41:31.125787 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:41:31.126168 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:41:31.125892 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:42:35.125869264 +0000 UTC m=+163.643680171 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:41:31.126168 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:41:31.125925 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:42:35.125912213 +0000 UTC m=+163.643723132 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:41:36.312614 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:41:36.312583 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gs8sb" Jul 22 18:42:01.740421 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:42:01.740364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:42:01.740940 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:01.740518 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 18:42:01.740940 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:01.740590 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:44:03.740573251 +0000 UTC m=+252.258384143 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : secret "metrics-daemon-secret" not found Jul 22 18:42:30.279279 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:30.279227 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" podUID="0c73ba3b-7767-41d8-9900-be044340308f" Jul 22 18:42:30.294590 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:30.294551 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ffc88" podUID="965c0244-14f9-4460-9e1a-62c696692420" Jul 22 18:42:30.510472 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:42:30.510443 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ffc88" Jul 22 18:42:30.510630 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:42:30.510443 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:42:32.058076 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:32.058037 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:42:35.160949 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:42:35.160910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:42:35.161363 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:42:35.160973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:42:35.161363 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:35.161060 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 18:42:35.161363 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:35.161114 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 18:42:35.161363 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:35.161129 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-785cd8c857-vhdvr: secret "image-registry-tls" not found Jul 22 18:42:35.161363 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:35.161142 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls podName:965c0244-14f9-4460-9e1a-62c696692420 nodeName:}" failed. No retries permitted until 2026-07-22 18:44:37.161122093 +0000 UTC m=+285.678933003 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls") pod "dns-default-ffc88" (UID: "965c0244-14f9-4460-9e1a-62c696692420") : secret "dns-default-metrics-tls" not found Jul 22 18:42:35.161363 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:42:35.161194 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls podName:0c73ba3b-7767-41d8-9900-be044340308f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:37.161174772 +0000 UTC m=+285.678985667 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls") pod "image-registry-785cd8c857-vhdvr" (UID: "0c73ba3b-7767-41d8-9900-be044340308f") : secret "image-registry-tls" not found Jul 22 18:42:43.037586 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:42:43.037540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:44:03.787850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:03.787799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:44:03.788323 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:03.787933 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 18:44:03.788323 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:03.787989 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs podName:cf81b03f-0e6d-48dc-bafa-95e60c0c50a9 nodeName:}" failed. No retries permitted until 2026-07-22 18:46:05.787973756 +0000 UTC m=+374.305784648 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs") pod "network-metrics-daemon-sjfmj" (UID: "cf81b03f-0e6d-48dc-bafa-95e60c0c50a9") : secret "metrics-daemon-secret" not found Jul 22 18:44:19.636037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.635953 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt"] Jul 22 18:44:19.637565 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.637549 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" Jul 22 18:44:19.640884 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.640862 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.641002 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.640900 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kjc9d\"" Jul 22 18:44:19.642451 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.642425 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.644970 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.644950 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp"] Jul 22 18:44:19.647695 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.647352 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9"] Jul 22 18:44:19.650367 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.650004 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.650927 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.650889 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-x5kfm"] Jul 22 18:44:19.651562 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.651369 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.653517 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.653203 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.653517 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.653204 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 18:44:19.653517 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.653276 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 18:44:19.653517 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.653378 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5b8f49c9b6-w6xm6"] Jul 22 18:44:19.653762 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.653653 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.653762 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.653758 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.654282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.654258 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-fcvj9\"" Jul 22 18:44:19.654566 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.654546 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-r9znd\"" Jul 22 18:44:19.655218 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.655201 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt"] Jul 22 18:44:19.655298 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.655260 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.657309 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.657292 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.657472 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.657364 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 18:44:19.657567 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.657488 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 18:44:19.657567 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.657457 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.657673 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.657377 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.657673 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.657479 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 18:44:19.658110 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658093 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 18:44:19.658244 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658181 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.658329 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658256 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lxjw8\"" Jul 22 18:44:19.658554 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658538 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 18:44:19.658699 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658678 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9"] Jul 22 18:44:19.658778 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658767 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 18:44:19.658851 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.658771 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.659316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.659211 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kfh92\"" Jul 22 18:44:19.659442 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.659424 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 18:44:19.659535 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.659515 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 18:44:19.659779 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.659758 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.666152 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.666130 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 18:44:19.666812 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.666787 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp"] Jul 22 18:44:19.668653 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.668631 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-x5kfm"] Jul 22 18:44:19.680163 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.680123 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b8f49c9b6-w6xm6"] Jul 22 18:44:19.737588 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.737553 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-skfzm"] Jul 22 18:44:19.739389 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.739372 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.740836 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.740798 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6"] Jul 22 18:44:19.741646 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.741628 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 18:44:19.741725 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.741685 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.741725 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.741629 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 18:44:19.742215 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.742194 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9hsd8\"" Jul 22 18:44:19.742357 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.742289 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.742487 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.742468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:19.746228 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.746131 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 18:44:19.746527 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.746501 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-wf9cl\"" Jul 22 18:44:19.746626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.746565 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 18:44:19.746626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.746572 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 18:44:19.747122 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.747100 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 18:44:19.748091 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.748069 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 18:44:19.755868 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.755846 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-skfzm"] Jul 22 18:44:19.756703 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.756679 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6"] Jul 22 18:44:19.798946 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.798910 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/844ec409-9225-487f-a44e-c8b6d7db9682-serving-cert\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.798946 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.798945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-stats-auth\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.799108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.798984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/844ec409-9225-487f-a44e-c8b6d7db9682-trusted-ca\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.799108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.799108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-default-certificate\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.799108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t57gw\" (UniqueName: \"kubernetes.io/projected/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-kube-api-access-t57gw\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.799263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.799263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799156 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-config\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.799263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.799263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwzg\" (UniqueName: \"kubernetes.io/projected/93d054bc-ebb9-4490-9fbc-cfe880af248f-kube-api-access-fbwzg\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.799263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/844ec409-9225-487f-a44e-c8b6d7db9682-config\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.800246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc4tw\" (UniqueName: \"kubernetes.io/projected/844ec409-9225-487f-a44e-c8b6d7db9682-kube-api-access-cc4tw\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.800246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.800246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.800246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799807 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9rkj\" (UniqueName: \"kubernetes.io/projected/e35897d3-8bde-4bd0-9e76-efbd0b6fd71e-kube-api-access-b9rkj\") pod \"volume-data-source-validator-78cf4c6f6-k5jtt\" (UID: \"e35897d3-8bde-4bd0-9e76-efbd0b6fd71e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" Jul 22 18:44:19.800246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.799858 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7rgk\" (UniqueName: \"kubernetes.io/projected/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-kube-api-access-h7rgk\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.900509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/844ec409-9225-487f-a44e-c8b6d7db9682-serving-cert\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.900509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900468 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-stats-auth\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.900509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/844ec409-9225-487f-a44e-c8b6d7db9682-trusted-ca\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff741955-344e-4739-98a4-91530fe2f3cf-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900549 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fbde1910-202e-40d4-87cc-fe1c3b781290-snapshots\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-default-certificate\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t57gw\" (UniqueName: \"kubernetes.io/projected/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-kube-api-access-t57gw\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900712 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-config\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900803 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbde1910-202e-40d4-87cc-fe1c3b781290-serving-cert\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:19.900813 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbde1910-202e-40d4-87cc-fe1c3b781290-tmp\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.900885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900872 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dkvg\" (UniqueName: \"kubernetes.io/projected/fbde1910-202e-40d4-87cc-fe1c3b781290-kube-api-access-5dkvg\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:19.900909 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:20.40088785 +0000 UTC m=+268.918698742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : secret "router-metrics-certs-default" not found Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwzg\" (UniqueName: \"kubernetes.io/projected/93d054bc-ebb9-4490-9fbc-cfe880af248f-kube-api-access-fbwzg\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.900983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/844ec409-9225-487f-a44e-c8b6d7db9682-config\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc4tw\" (UniqueName: \"kubernetes.io/projected/844ec409-9225-487f-a44e-c8b6d7db9682-kube-api-access-cc4tw\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbde1910-202e-40d4-87cc-fe1c3b781290-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901108 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9rkj\" (UniqueName: \"kubernetes.io/projected/e35897d3-8bde-4bd0-9e76-efbd0b6fd71e-kube-api-access-b9rkj\") pod \"volume-data-source-validator-78cf4c6f6-k5jtt\" (UID: \"e35897d3-8bde-4bd0-9e76-efbd0b6fd71e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7rgk\" (UniqueName: \"kubernetes.io/projected/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-kube-api-access-h7rgk\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbde1910-202e-40d4-87cc-fe1c3b781290-service-ca-bundle\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfmqj\" (UniqueName: \"kubernetes.io/projected/ff741955-344e-4739-98a4-91530fe2f3cf-kube-api-access-gfmqj\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901335 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-config\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.901503 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:19.901485 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:20.401470054 +0000 UTC m=+268.919280962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : configmap references non-existent config key: service-ca.crt Jul 22 18:44:19.902233 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.901545 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/844ec409-9225-487f-a44e-c8b6d7db9682-trusted-ca\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.902233 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.902158 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/844ec409-9225-487f-a44e-c8b6d7db9682-config\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.902533 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.902507 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.903446 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.903422 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/844ec409-9225-487f-a44e-c8b6d7db9682-serving-cert\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.903530 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.903447 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.903565 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.903540 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-default-certificate\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.903654 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.903639 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-serving-cert\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.903780 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.903759 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-stats-auth\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.912938 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.912918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwzg\" (UniqueName: \"kubernetes.io/projected/93d054bc-ebb9-4490-9fbc-cfe880af248f-kube-api-access-fbwzg\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:19.913135 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.913113 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc4tw\" (UniqueName: \"kubernetes.io/projected/844ec409-9225-487f-a44e-c8b6d7db9682-kube-api-access-cc4tw\") pod \"console-operator-565b7b569d-x5kfm\" (UID: \"844ec409-9225-487f-a44e-c8b6d7db9682\") " pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:19.913692 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.913665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t57gw\" (UniqueName: \"kubernetes.io/projected/2722a962-6658-4f2a-a0ff-2bbfd3cec7e1-kube-api-access-t57gw\") pod \"service-ca-operator-5bcbc94c78-jkzgp\" (UID: \"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.914086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.914067 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9rkj\" (UniqueName: \"kubernetes.io/projected/e35897d3-8bde-4bd0-9e76-efbd0b6fd71e-kube-api-access-b9rkj\") pod \"volume-data-source-validator-78cf4c6f6-k5jtt\" (UID: \"e35897d3-8bde-4bd0-9e76-efbd0b6fd71e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" Jul 22 18:44:19.914615 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.914597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7rgk\" (UniqueName: \"kubernetes.io/projected/1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63-kube-api-access-h7rgk\") pod \"kube-storage-version-migrator-operator-7658d9b749-kcvp9\" (UID: \"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.947625 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.947608 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" Jul 22 18:44:19.963167 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.963146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" Jul 22 18:44:19.971060 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.971035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" Jul 22 18:44:19.976795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:19.976770 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:20.002001 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.001930 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbde1910-202e-40d4-87cc-fe1c3b781290-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.002001 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.001976 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:20.002143 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbde1910-202e-40d4-87cc-fe1c3b781290-service-ca-bundle\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.002143 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfmqj\" (UniqueName: \"kubernetes.io/projected/ff741955-344e-4739-98a4-91530fe2f3cf-kube-api-access-gfmqj\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:20.002143 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002108 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff741955-344e-4739-98a4-91530fe2f3cf-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:20.002299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fbde1910-202e-40d4-87cc-fe1c3b781290-snapshots\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.002299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbde1910-202e-40d4-87cc-fe1c3b781290-serving-cert\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.002299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002237 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbde1910-202e-40d4-87cc-fe1c3b781290-tmp\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.002299 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.002260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dkvg\" (UniqueName: \"kubernetes.io/projected/fbde1910-202e-40d4-87cc-fe1c3b781290-kube-api-access-5dkvg\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.004719 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.003717 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbde1910-202e-40d4-87cc-fe1c3b781290-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.004719 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.003745 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ff741955-344e-4739-98a4-91530fe2f3cf-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:20.004719 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.003884 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:20.004719 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.003958 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls podName:ff741955-344e-4739-98a4-91530fe2f3cf nodeName:}" failed. No retries permitted until 2026-07-22 18:44:20.503939122 +0000 UTC m=+269.021750039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-vwsv6" (UID: "ff741955-344e-4739-98a4-91530fe2f3cf") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:20.004719 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.004133 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbde1910-202e-40d4-87cc-fe1c3b781290-tmp\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.004719 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.004592 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbde1910-202e-40d4-87cc-fe1c3b781290-service-ca-bundle\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.005281 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.005224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fbde1910-202e-40d4-87cc-fe1c3b781290-snapshots\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.008641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.008567 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbde1910-202e-40d4-87cc-fe1c3b781290-serving-cert\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.011011 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.010855 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dkvg\" (UniqueName: \"kubernetes.io/projected/fbde1910-202e-40d4-87cc-fe1c3b781290-kube-api-access-5dkvg\") pod \"insights-operator-555d9d7885-skfzm\" (UID: \"fbde1910-202e-40d4-87cc-fe1c3b781290\") " pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.012303 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.012266 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfmqj\" (UniqueName: \"kubernetes.io/projected/ff741955-344e-4739-98a4-91530fe2f3cf-kube-api-access-gfmqj\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:20.051347 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.051110 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-skfzm" Jul 22 18:44:20.105482 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.105441 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt"] Jul 22 18:44:20.122602 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.122568 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp"] Jul 22 18:44:20.147631 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.147607 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9"] Jul 22 18:44:20.152384 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:20.152324 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf0d3c9_3ea6_4b3f_b8fd_ca5bd6e77d63.slice/crio-19848a782eee85e303dc789223f4bfa3ae3f721244ef0c4d2dc3638b70329bc4 WatchSource:0}: Error finding container 19848a782eee85e303dc789223f4bfa3ae3f721244ef0c4d2dc3638b70329bc4: Status 404 returned error can't find the container with id 19848a782eee85e303dc789223f4bfa3ae3f721244ef0c4d2dc3638b70329bc4 Jul 22 18:44:20.162825 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.162789 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-x5kfm"] Jul 22 18:44:20.165589 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:20.165566 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod844ec409_9225_487f_a44e_c8b6d7db9682.slice/crio-9029a1e054042b9e94361e5d2f2693d41606deea19a5504f2b31552ec2e3b582 WatchSource:0}: Error finding container 9029a1e054042b9e94361e5d2f2693d41606deea19a5504f2b31552ec2e3b582: Status 404 returned error can't find the container with id 9029a1e054042b9e94361e5d2f2693d41606deea19a5504f2b31552ec2e3b582 Jul 22 18:44:20.197235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.197133 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-skfzm"] Jul 22 18:44:20.199329 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:20.199309 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbde1910_202e_40d4_87cc_fe1c3b781290.slice/crio-035d6934cc538e220c3a1940f9d61fc5d3ef2eb4e8c6d46f88d8243adc2aeac6 WatchSource:0}: Error finding container 035d6934cc538e220c3a1940f9d61fc5d3ef2eb4e8c6d46f88d8243adc2aeac6: Status 404 returned error can't find the container with id 035d6934cc538e220c3a1940f9d61fc5d3ef2eb4e8c6d46f88d8243adc2aeac6 Jul 22 18:44:20.407587 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.407496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:20.407587 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.407578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:20.407793 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.407674 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:21.40765562 +0000 UTC m=+269.925466513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : configmap references non-existent config key: service-ca.crt Jul 22 18:44:20.407793 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.407675 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 18:44:20.407793 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.407714 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:21.40770703 +0000 UTC m=+269.925517922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : secret "router-metrics-certs-default" not found Jul 22 18:44:20.509150 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.509112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:20.509315 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.509225 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:20.509315 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:20.509282 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls podName:ff741955-344e-4739-98a4-91530fe2f3cf nodeName:}" failed. No retries permitted until 2026-07-22 18:44:21.509268342 +0000 UTC m=+270.027079234 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-vwsv6" (UID: "ff741955-344e-4739-98a4-91530fe2f3cf") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:20.762980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.762840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-skfzm" event={"ID":"fbde1910-202e-40d4-87cc-fe1c3b781290","Type":"ContainerStarted","Data":"035d6934cc538e220c3a1940f9d61fc5d3ef2eb4e8c6d46f88d8243adc2aeac6"} Jul 22 18:44:20.764625 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.764590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" event={"ID":"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63","Type":"ContainerStarted","Data":"19848a782eee85e303dc789223f4bfa3ae3f721244ef0c4d2dc3638b70329bc4"} Jul 22 18:44:20.766302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.766266 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" event={"ID":"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1","Type":"ContainerStarted","Data":"774cb9f7974c74756f5d07b4d4120aad2cc752b678cd98bdfaa1224a631c27e5"} Jul 22 18:44:20.767828 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.767777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" event={"ID":"844ec409-9225-487f-a44e-c8b6d7db9682","Type":"ContainerStarted","Data":"9029a1e054042b9e94361e5d2f2693d41606deea19a5504f2b31552ec2e3b582"} Jul 22 18:44:20.769245 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:20.769218 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" event={"ID":"e35897d3-8bde-4bd0-9e76-efbd0b6fd71e","Type":"ContainerStarted","Data":"487dc306a673f467cdf8bdf4c9e85ce9494845f0d02828c428b20e76a0bd8828"} Jul 22 18:44:21.417804 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:21.417766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:21.417995 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:21.417866 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:21.418446 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:21.418066 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:23.418041598 +0000 UTC m=+271.935852507 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : configmap references non-existent config key: service-ca.crt Jul 22 18:44:21.418446 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:21.418183 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 18:44:21.418446 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:21.418253 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:23.418234862 +0000 UTC m=+271.936045760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : secret "router-metrics-certs-default" not found Jul 22 18:44:21.519207 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:21.518610 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:21.519207 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:21.518792 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:21.519207 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:21.518873 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls podName:ff741955-344e-4739-98a4-91530fe2f3cf nodeName:}" failed. No retries permitted until 2026-07-22 18:44:23.518853161 +0000 UTC m=+272.036664056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-vwsv6" (UID: "ff741955-344e-4739-98a4-91530fe2f3cf") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:22.777928 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:22.777881 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" event={"ID":"e35897d3-8bde-4bd0-9e76-efbd0b6fd71e","Type":"ContainerStarted","Data":"84eca6424a250835a93aa9e9197019ed54f5400e0997695fc25a269af10d9763"} Jul 22 18:44:22.795269 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:22.795205 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-k5jtt" podStartSLOduration=2.178686761 podStartE2EDuration="3.795182964s" totalImagesPullingTime="1.616496203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:22.794523172 +0000 UTC m=+271.312334087" watchObservedRunningTime="2026-07-22 18:44:22.795182964 +0000 UTC m=+271.312993879" Jul 22 18:44:23.440013 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:23.439968 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:23.440208 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:23.440044 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:23.440208 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:23.440125 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 18:44:23.440208 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:23.440193 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:27.440175766 +0000 UTC m=+275.957986665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : secret "router-metrics-certs-default" not found Jul 22 18:44:23.440370 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:23.440213 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:27.440204594 +0000 UTC m=+275.958015486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : configmap references non-existent config key: service-ca.crt Jul 22 18:44:23.540990 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:23.540957 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:23.541167 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:23.541145 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:23.541236 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:23.541227 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls podName:ff741955-344e-4739-98a4-91530fe2f3cf nodeName:}" failed. No retries permitted until 2026-07-22 18:44:27.541205038 +0000 UTC m=+276.059015931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-vwsv6" (UID: "ff741955-344e-4739-98a4-91530fe2f3cf") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:24.786099 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.786056 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-skfzm" event={"ID":"fbde1910-202e-40d4-87cc-fe1c3b781290","Type":"ContainerStarted","Data":"597641f091850fc0b56e727c7759513049a9d12e422c0077592ecb5fa0e03e16"} Jul 22 18:44:24.787727 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.787696 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" event={"ID":"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63","Type":"ContainerStarted","Data":"427ae76760c763e3cf44a7f91e7b34ba2e4ea06c507064de2831356ef29b1f21"} Jul 22 18:44:24.789240 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.789202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" event={"ID":"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1","Type":"ContainerStarted","Data":"661c4d5651a239773e4136fd5467de18e82bbbb37dbcfe338705330d2ea57be5"} Jul 22 18:44:24.790948 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.790927 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/0.log" Jul 22 18:44:24.791053 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.790968 2566 generic.go:356] "Generic (PLEG): container finished" podID="844ec409-9225-487f-a44e-c8b6d7db9682" containerID="40ba3e2ed4d4dc7b5e276fd418d08922119808b29347e1365e15c6ff9c23106c" exitCode=255 Jul 22 18:44:24.791053 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.791009 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" event={"ID":"844ec409-9225-487f-a44e-c8b6d7db9682","Type":"ContainerDied","Data":"40ba3e2ed4d4dc7b5e276fd418d08922119808b29347e1365e15c6ff9c23106c"} Jul 22 18:44:24.791239 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.791222 2566 scope.go:117] "RemoveContainer" containerID="40ba3e2ed4d4dc7b5e276fd418d08922119808b29347e1365e15c6ff9c23106c" Jul 22 18:44:24.802180 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.802129 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-skfzm" podStartSLOduration=1.9486362160000001 podStartE2EDuration="5.802109425s" totalImagesPullingTime="3.853473209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:24.801369378 +0000 UTC m=+273.319180316" watchObservedRunningTime="2026-07-22 18:44:24.802109425 +0000 UTC m=+273.319920343" Jul 22 18:44:24.816629 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.816585 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" podStartSLOduration=1.889762137 podStartE2EDuration="5.816569566s" totalImagesPullingTime="3.926807429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:24.815359736 +0000 UTC m=+273.333170651" watchObservedRunningTime="2026-07-22 18:44:24.816569566 +0000 UTC m=+273.334380479" Jul 22 18:44:24.832270 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:24.832209 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" podStartSLOduration=1.9330250169999998 podStartE2EDuration="5.832187792s" totalImagesPullingTime="3.899162775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:24.831496624 +0000 UTC m=+273.349307539" watchObservedRunningTime="2026-07-22 18:44:24.832187792 +0000 UTC m=+273.349998709" Jul 22 18:44:25.299368 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.299330 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mhpbl"] Jul 22 18:44:25.302696 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.302667 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.305033 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.305008 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 18:44:25.305162 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.305044 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 18:44:25.305339 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.305323 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rvpd8\"" Jul 22 18:44:25.318145 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.318114 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mhpbl"] Jul 22 18:44:25.357417 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.357381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6557j\" (UniqueName: \"kubernetes.io/projected/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-api-access-6557j\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.357566 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.357489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.357566 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.357520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.357566 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.357541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-data-volume\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.357695 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.357674 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-crio-socket\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458195 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458160 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-crio-socket\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458351 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6557j\" (UniqueName: \"kubernetes.io/projected/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-api-access-6557j\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458351 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458324 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-crio-socket\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458462 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458497 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458530 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458499 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-data-volume\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.458571 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.458555 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:44:25.458629 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.458613 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:44:25.458681 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.458620 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:25.958598584 +0000 UTC m=+274.476409483 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : configmap "kube-rbac-proxy" not found Jul 22 18:44:25.458725 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.458706 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:25.958682345 +0000 UTC m=+274.476493242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : secret "insights-runtime-extractor-tls" not found Jul 22 18:44:25.458853 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.458808 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-data-volume\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.466266 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.466236 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6557j\" (UniqueName: \"kubernetes.io/projected/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-api-access-6557j\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.796270 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.796240 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:44:25.796667 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.796633 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/0.log" Jul 22 18:44:25.796708 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.796665 2566 generic.go:356] "Generic (PLEG): container finished" podID="844ec409-9225-487f-a44e-c8b6d7db9682" containerID="c35f708b21711b52b835ecf9d905b2f0868d6a1a2e1d7f44c4b29fed60cebf2e" exitCode=255 Jul 22 18:44:25.796791 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.796760 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" event={"ID":"844ec409-9225-487f-a44e-c8b6d7db9682","Type":"ContainerDied","Data":"c35f708b21711b52b835ecf9d905b2f0868d6a1a2e1d7f44c4b29fed60cebf2e"} Jul 22 18:44:25.796931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.796848 2566 scope.go:117] "RemoveContainer" containerID="40ba3e2ed4d4dc7b5e276fd418d08922119808b29347e1365e15c6ff9c23106c" Jul 22 18:44:25.797014 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.796996 2566 scope.go:117] "RemoveContainer" containerID="c35f708b21711b52b835ecf9d905b2f0868d6a1a2e1d7f44c4b29fed60cebf2e" Jul 22 18:44:25.797284 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.797259 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-x5kfm_openshift-console-operator(844ec409-9225-487f-a44e-c8b6d7db9682)\"" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" podUID="844ec409-9225-487f-a44e-c8b6d7db9682" Jul 22 18:44:25.963380 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.963343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.963380 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:25.963384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:25.963591 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.963453 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:44:25.963591 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.963472 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:44:25.963591 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.963523 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:26.96350544 +0000 UTC m=+275.481316332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : configmap "kube-rbac-proxy" not found Jul 22 18:44:25.963591 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:25.963536 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:26.963530157 +0000 UTC m=+275.481341048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : secret "insights-runtime-extractor-tls" not found Jul 22 18:44:26.800564 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:26.800540 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:44:26.801056 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:26.800961 2566 scope.go:117] "RemoveContainer" containerID="c35f708b21711b52b835ecf9d905b2f0868d6a1a2e1d7f44c4b29fed60cebf2e" Jul 22 18:44:26.801165 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:26.801141 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-x5kfm_openshift-console-operator(844ec409-9225-487f-a44e-c8b6d7db9682)\"" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" podUID="844ec409-9225-487f-a44e-c8b6d7db9682" Jul 22 18:44:26.971700 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:26.971661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:26.971700 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:26.971705 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:26.971974 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:26.971790 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:44:26.971974 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:26.971848 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:44:26.971974 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:26.971878 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:28.971860324 +0000 UTC m=+277.489671217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : configmap "kube-rbac-proxy" not found Jul 22 18:44:26.971974 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:26.971911 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:28.971895898 +0000 UTC m=+277.489706805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : secret "insights-runtime-extractor-tls" not found Jul 22 18:44:27.476890 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:27.476857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:27.477072 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:27.476924 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:27.477072 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:27.476999 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 18:44:27.477072 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:27.477064 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:35.477045755 +0000 UTC m=+283.994856648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : secret "router-metrics-certs-default" not found Jul 22 18:44:27.477249 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:27.477082 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:35.477075882 +0000 UTC m=+283.994886774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : configmap references non-existent config key: service-ca.crt Jul 22 18:44:27.578227 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:27.578196 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:27.578382 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:27.578304 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:27.578382 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:27.578349 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls podName:ff741955-344e-4739-98a4-91530fe2f3cf nodeName:}" failed. No retries permitted until 2026-07-22 18:44:35.578337827 +0000 UTC m=+284.096148719 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-vwsv6" (UID: "ff741955-344e-4739-98a4-91530fe2f3cf") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:28.082995 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:28.082968 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bwrkx_422501a3-4f95-45c0-b601-33d2e4410917/dns-node-resolver/0.log" Jul 22 18:44:28.989374 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:28.989332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:28.989580 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:28.989381 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:28.989580 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:28.989498 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:44:28.989580 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:28.989574 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:44:28.989743 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:28.989583 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:32.989562738 +0000 UTC m=+281.507373642 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : configmap "kube-rbac-proxy" not found Jul 22 18:44:28.989743 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:28.989624 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:32.989611051 +0000 UTC m=+281.507421947 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : secret "insights-runtime-extractor-tls" not found Jul 22 18:44:29.083835 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:29.083794 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8mls5_65c4db89-8088-48e2-9bc3-ab21f0c74fef/node-ca/0.log" Jul 22 18:44:29.977780 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:29.977689 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:29.977780 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:29.977786 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:29.978149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:29.978136 2566 scope.go:117] "RemoveContainer" containerID="c35f708b21711b52b835ecf9d905b2f0868d6a1a2e1d7f44c4b29fed60cebf2e" Jul 22 18:44:29.978290 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:29.978273 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-x5kfm_openshift-console-operator(844ec409-9225-487f-a44e-c8b6d7db9682)\"" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" podUID="844ec409-9225-487f-a44e-c8b6d7db9682" Jul 22 18:44:30.084144 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:30.084120 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdkfp_71b46f3d-ad01-4915-91d3-03d2e769d731/serve-healthcheck-canary/0.log" Jul 22 18:44:30.685885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:30.685846 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-kcvp9_1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63/kube-storage-version-migrator-operator/0.log" Jul 22 18:44:33.018028 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:33.017993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:33.018028 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:33.018030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:33.018465 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:33.018115 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 18:44:33.018465 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:33.018158 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:44:33.018465 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:33.018177 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:41.018161666 +0000 UTC m=+289.535972557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : configmap "kube-rbac-proxy" not found Jul 22 18:44:33.018465 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:33.018199 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:41.018188318 +0000 UTC m=+289.535999210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : secret "insights-runtime-extractor-tls" not found Jul 22 18:44:33.511586 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:33.511545 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ffc88" podUID="965c0244-14f9-4460-9e1a-62c696692420" Jul 22 18:44:33.511749 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:33.511545 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" podUID="0c73ba3b-7767-41d8-9900-be044340308f" Jul 22 18:44:33.823904 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:33.823164 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:44:33.823904 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:33.823623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ffc88" Jul 22 18:44:35.539685 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:35.539649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:35.540144 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:35.539726 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:35.540144 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:35.539845 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 18:44:35.540144 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:35.539904 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:51.539880508 +0000 UTC m=+300.057691409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : configmap references non-existent config key: service-ca.crt Jul 22 18:44:35.540144 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:35.539934 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs podName:93d054bc-ebb9-4490-9fbc-cfe880af248f nodeName:}" failed. No retries permitted until 2026-07-22 18:44:51.539923553 +0000 UTC m=+300.057734449 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs") pod "router-default-5b8f49c9b6-w6xm6" (UID: "93d054bc-ebb9-4490-9fbc-cfe880af248f") : secret "router-metrics-certs-default" not found Jul 22 18:44:35.640192 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:35.640158 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:35.640342 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:35.640300 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:35.640390 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:35.640353 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls podName:ff741955-344e-4739-98a4-91530fe2f3cf nodeName:}" failed. No retries permitted until 2026-07-22 18:44:51.640338634 +0000 UTC m=+300.158149527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-vwsv6" (UID: "ff741955-344e-4739-98a4-91530fe2f3cf") : secret "cluster-monitoring-operator-tls" not found Jul 22 18:44:37.253673 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.253630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:44:37.254167 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.253688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:44:37.256137 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.256107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/965c0244-14f9-4460-9e1a-62c696692420-metrics-tls\") pod \"dns-default-ffc88\" (UID: \"965c0244-14f9-4460-9e1a-62c696692420\") " pod="openshift-dns/dns-default-ffc88" Jul 22 18:44:37.256285 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.256265 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"image-registry-785cd8c857-vhdvr\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:44:37.427715 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.427677 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hbkg4\"" Jul 22 18:44:37.427715 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.427683 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh2s\"" Jul 22 18:44:37.435602 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.435580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ffc88" Jul 22 18:44:37.435723 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.435693 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:44:37.567369 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.567327 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ffc88"] Jul 22 18:44:37.570848 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:37.570797 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965c0244_14f9_4460_9e1a_62c696692420.slice/crio-cab02263d2075dbc0c9e93be36080bd26aed38532c79ff302b88116bb5e204bb WatchSource:0}: Error finding container cab02263d2075dbc0c9e93be36080bd26aed38532c79ff302b88116bb5e204bb: Status 404 returned error can't find the container with id cab02263d2075dbc0c9e93be36080bd26aed38532c79ff302b88116bb5e204bb Jul 22 18:44:37.587012 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.586991 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-785cd8c857-vhdvr"] Jul 22 18:44:37.594927 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:37.594905 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c73ba3b_7767_41d8_9900_be044340308f.slice/crio-baee53dd8b35ae9d55abafce192ce13cbb74c5b9e67d84ab17b5474b34d1efa1 WatchSource:0}: Error finding container baee53dd8b35ae9d55abafce192ce13cbb74c5b9e67d84ab17b5474b34d1efa1: Status 404 returned error can't find the container with id baee53dd8b35ae9d55abafce192ce13cbb74c5b9e67d84ab17b5474b34d1efa1 Jul 22 18:44:37.834805 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.834716 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ffc88" event={"ID":"965c0244-14f9-4460-9e1a-62c696692420","Type":"ContainerStarted","Data":"cab02263d2075dbc0c9e93be36080bd26aed38532c79ff302b88116bb5e204bb"} Jul 22 18:44:37.836270 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.836239 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" event={"ID":"0c73ba3b-7767-41d8-9900-be044340308f","Type":"ContainerStarted","Data":"00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9"} Jul 22 18:44:37.836408 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.836287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" event={"ID":"0c73ba3b-7767-41d8-9900-be044340308f","Type":"ContainerStarted","Data":"baee53dd8b35ae9d55abafce192ce13cbb74c5b9e67d84ab17b5474b34d1efa1"} Jul 22 18:44:37.836408 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.836397 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:44:37.861716 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:37.861672 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" podStartSLOduration=273.861659233 podStartE2EDuration="4m33.861659233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:37.860664441 +0000 UTC m=+286.378475355" watchObservedRunningTime="2026-07-22 18:44:37.861659233 +0000 UTC m=+286.379470125" Jul 22 18:44:39.847779 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:39.847742 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ffc88" event={"ID":"965c0244-14f9-4460-9e1a-62c696692420","Type":"ContainerStarted","Data":"33cc358424fbbbb392dfcaba18c7acc25d0c5535e3cbb2a8e1c71e065b14a0ec"} Jul 22 18:44:39.848142 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:39.847847 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ffc88" event={"ID":"965c0244-14f9-4460-9e1a-62c696692420","Type":"ContainerStarted","Data":"bf3de5b197bfa1193545c9e34e79fc29fc6e77991cf2a39a51e345c65c3af3e4"} Jul 22 18:44:39.848142 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:39.847877 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ffc88" Jul 22 18:44:40.073415 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:40.073366 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ffc88" podStartSLOduration=251.283100662 podStartE2EDuration="4m13.073350752s" totalImagesPullingTime="1.79025009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:40:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:39.862808235 +0000 UTC m=+288.380619188" watchObservedRunningTime="2026-07-22 18:44:40.073350752 +0000 UTC m=+288.591161711" Jul 22 18:44:41.088763 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:41.088732 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:41.088763 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:41.088766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:41.089268 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:41.088904 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 18:44:41.089268 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:41.088958 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls podName:3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c nodeName:}" failed. No retries permitted until 2026-07-22 18:44:57.088939663 +0000 UTC m=+305.606750556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mhpbl" (UID: "3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c") : secret "insights-runtime-extractor-tls" not found Jul 22 18:44:41.089268 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:41.089182 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:45.038515 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:45.038483 2566 scope.go:117] "RemoveContainer" containerID="c35f708b21711b52b835ecf9d905b2f0868d6a1a2e1d7f44c4b29fed60cebf2e" Jul 22 18:44:45.868705 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:45.868679 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:44:45.868886 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:45.868742 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" event={"ID":"844ec409-9225-487f-a44e-c8b6d7db9682","Type":"ContainerStarted","Data":"7dcf415be0375ab3a9bfbfec29765ccf33c2fc2c98b15764384aa6e872707141"} Jul 22 18:44:45.869058 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:45.869034 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:45.873657 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:45.873637 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" Jul 22 18:44:45.887451 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:45.887405 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-x5kfm" podStartSLOduration=22.997968816 podStartE2EDuration="26.887391421s" totalImagesPullingTime="3.889422605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:45.886544059 +0000 UTC m=+294.404355009" watchObservedRunningTime="2026-07-22 18:44:45.887391421 +0000 UTC m=+294.405202335" Jul 22 18:44:46.038789 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:44:46.038752 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-sjfmj" podUID="cf81b03f-0e6d-48dc-bafa-95e60c0c50a9" Jul 22 18:44:47.887773 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.887739 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-tx8r5"] Jul 22 18:44:47.892952 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.892924 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:44:47.893649 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.893629 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-785cd8c857-vhdvr"] Jul 22 18:44:47.895141 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.895109 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 18:44:47.895349 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.895334 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 18:44:47.896547 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.896522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-gv2js\"" Jul 22 18:44:47.907788 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:47.907762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-tx8r5"] Jul 22 18:44:48.046308 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:48.046279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75mzl\" (UniqueName: \"kubernetes.io/projected/9460965c-3154-4b38-bfd3-dfee19bd240d-kube-api-access-75mzl\") pod \"downloads-5bc67dcd46-tx8r5\" (UID: \"9460965c-3154-4b38-bfd3-dfee19bd240d\") " pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:44:48.147244 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:48.147155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75mzl\" (UniqueName: \"kubernetes.io/projected/9460965c-3154-4b38-bfd3-dfee19bd240d-kube-api-access-75mzl\") pod \"downloads-5bc67dcd46-tx8r5\" (UID: \"9460965c-3154-4b38-bfd3-dfee19bd240d\") " pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:44:48.160908 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:48.160880 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75mzl\" (UniqueName: \"kubernetes.io/projected/9460965c-3154-4b38-bfd3-dfee19bd240d-kube-api-access-75mzl\") pod \"downloads-5bc67dcd46-tx8r5\" (UID: \"9460965c-3154-4b38-bfd3-dfee19bd240d\") " pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:44:48.203278 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:48.203240 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:44:48.333433 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:48.333405 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-tx8r5"] Jul 22 18:44:48.336750 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:48.336715 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9460965c_3154_4b38_bfd3_dfee19bd240d.slice/crio-3ecdc67e6fa9e0367b2f800b91c389bdd5ecf1403ed12916e0fbc910e54a83ed WatchSource:0}: Error finding container 3ecdc67e6fa9e0367b2f800b91c389bdd5ecf1403ed12916e0fbc910e54a83ed: Status 404 returned error can't find the container with id 3ecdc67e6fa9e0367b2f800b91c389bdd5ecf1403ed12916e0fbc910e54a83ed Jul 22 18:44:48.880342 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:48.880300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-tx8r5" event={"ID":"9460965c-3154-4b38-bfd3-dfee19bd240d","Type":"ContainerStarted","Data":"3ecdc67e6fa9e0367b2f800b91c389bdd5ecf1403ed12916e0fbc910e54a83ed"} Jul 22 18:44:49.854166 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:49.854136 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ffc88" Jul 22 18:44:51.575379 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.575337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:51.575845 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.575527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:51.576333 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.576303 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93d054bc-ebb9-4490-9fbc-cfe880af248f-service-ca-bundle\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:51.578111 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.578085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/93d054bc-ebb9-4490-9fbc-cfe880af248f-metrics-certs\") pod \"router-default-5b8f49c9b6-w6xm6\" (UID: \"93d054bc-ebb9-4490-9fbc-cfe880af248f\") " pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:51.676325 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.676287 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:51.679112 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.679084 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ff741955-344e-4739-98a4-91530fe2f3cf-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-vwsv6\" (UID: \"ff741955-344e-4739-98a4-91530fe2f3cf\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:51.781411 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.781382 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:51.857670 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.857581 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" Jul 22 18:44:51.917586 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.917541 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:44:51.918094 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.918070 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:44:51.927944 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.927925 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:44:51.929228 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.929200 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:44:51.937616 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.937594 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 18:44:51.950064 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:51.949952 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b8f49c9b6-w6xm6"] Jul 22 18:44:51.955549 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:51.955526 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d054bc_ebb9_4490_9fbc_cfe880af248f.slice/crio-15e039ca265a11d9660261506aaa5e2358cbbca3e15996287f91098ec4fd82c8 WatchSource:0}: Error finding container 15e039ca265a11d9660261506aaa5e2358cbbca3e15996287f91098ec4fd82c8: Status 404 returned error can't find the container with id 15e039ca265a11d9660261506aaa5e2358cbbca3e15996287f91098ec4fd82c8 Jul 22 18:44:52.024352 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:52.024156 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6"] Jul 22 18:44:52.036765 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:52.036733 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff741955_344e_4739_98a4_91530fe2f3cf.slice/crio-cf4edda85400f5dd0a9f46e5f9cb404bb749229033e6834ff9919621e174e790 WatchSource:0}: Error finding container cf4edda85400f5dd0a9f46e5f9cb404bb749229033e6834ff9919621e174e790: Status 404 returned error can't find the container with id cf4edda85400f5dd0a9f46e5f9cb404bb749229033e6834ff9919621e174e790 Jul 22 18:44:52.909768 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:52.909730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" event={"ID":"ff741955-344e-4739-98a4-91530fe2f3cf","Type":"ContainerStarted","Data":"cf4edda85400f5dd0a9f46e5f9cb404bb749229033e6834ff9919621e174e790"} Jul 22 18:44:52.911619 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:52.911362 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" event={"ID":"93d054bc-ebb9-4490-9fbc-cfe880af248f","Type":"ContainerStarted","Data":"55db4fc14bd4efa41a4c68ae24094d9dc1c2a6ab61ffd0a2213374446180b7c5"} Jul 22 18:44:52.911619 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:52.911413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" event={"ID":"93d054bc-ebb9-4490-9fbc-cfe880af248f","Type":"ContainerStarted","Data":"15e039ca265a11d9660261506aaa5e2358cbbca3e15996287f91098ec4fd82c8"} Jul 22 18:44:52.931452 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:52.931402 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" podStartSLOduration=33.931384948 podStartE2EDuration="33.931384948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:52.930158285 +0000 UTC m=+301.447969219" watchObservedRunningTime="2026-07-22 18:44:52.931384948 +0000 UTC m=+301.449195862" Jul 22 18:44:53.782427 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:53.782390 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:53.785262 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:53.785237 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:53.915315 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:53.915280 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:53.917063 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:53.916805 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5b8f49c9b6-w6xm6" Jul 22 18:44:54.731405 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.731372 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c8ccf648c-8f6pb"] Jul 22 18:44:54.734628 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.734602 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:54.737832 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.737592 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 18:44:54.737832 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.737683 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 18:44:54.737832 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.737737 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 18:44:54.737832 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.737592 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 18:44:54.737832 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.737759 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-7zj7v\"" Jul 22 18:44:54.737832 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.737685 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 18:44:54.747653 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.747602 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8ccf648c-8f6pb"] Jul 22 18:44:54.907261 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.907222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-serving-cert\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:54.907261 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.907263 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-oauth-serving-cert\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:54.907497 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.907291 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-oauth-config\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:54.907497 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.907381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmdq4\" (UniqueName: \"kubernetes.io/projected/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-kube-api-access-pmdq4\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:54.907497 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.907439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-service-ca\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:54.907614 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:54.907549 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-config\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.008318 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.008230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-config\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.008318 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.008282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-serving-cert\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.008318 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.008314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-oauth-serving-cert\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.008318 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.008337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-oauth-config\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.009081 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.008370 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmdq4\" (UniqueName: \"kubernetes.io/projected/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-kube-api-access-pmdq4\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.009081 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.008396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-service-ca\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.009192 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.009098 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-config\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.009192 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.009121 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-service-ca\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.009668 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.009621 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-oauth-serving-cert\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.011922 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.011878 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-serving-cert\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.012499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.012478 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-oauth-config\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.019024 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.018964 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmdq4\" (UniqueName: \"kubernetes.io/projected/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-kube-api-access-pmdq4\") pod \"console-7c8ccf648c-8f6pb\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.046862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.046736 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:44:55.338707 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.338669 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c8ccf648c-8f6pb"] Jul 22 18:44:55.342366 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:55.342338 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8a0f24_1d15_4c5d_95cd_6150abf66f8b.slice/crio-faf16f92a833e99e1f21e5e6c9396eefbba6f2c62f4978cbf65aaa9974907d3a WatchSource:0}: Error finding container faf16f92a833e99e1f21e5e6c9396eefbba6f2c62f4978cbf65aaa9974907d3a: Status 404 returned error can't find the container with id faf16f92a833e99e1f21e5e6c9396eefbba6f2c62f4978cbf65aaa9974907d3a Jul 22 18:44:55.344941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.344902 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 18:44:55.924686 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.924608 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccf648c-8f6pb" event={"ID":"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b","Type":"ContainerStarted","Data":"faf16f92a833e99e1f21e5e6c9396eefbba6f2c62f4978cbf65aaa9974907d3a"} Jul 22 18:44:55.926977 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.926933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" event={"ID":"ff741955-344e-4739-98a4-91530fe2f3cf","Type":"ContainerStarted","Data":"c7fb03bb042e6d24f5e1e21abcf306468669037c31f355c4f1197b67637acfa6"} Jul 22 18:44:55.946934 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:55.946878 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-vwsv6" podStartSLOduration=33.757828539 podStartE2EDuration="36.946857234s" totalImagesPullingTime="3.189028695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:55.946138374 +0000 UTC m=+304.463949312" watchObservedRunningTime="2026-07-22 18:44:55.946857234 +0000 UTC m=+304.464668149" Jul 22 18:44:57.117221 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.117187 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cf6bf6968-l4jrk"] Jul 22 18:44:57.119475 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.119449 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.127941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.127029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:57.127941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.127829 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 18:44:57.129788 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.129769 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cf6bf6968-l4jrk"] Jul 22 18:44:57.131784 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.131740 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mhpbl\" (UID: \"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c\") " pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:57.228212 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.228174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-config\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.228397 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.228222 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-service-ca\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.228468 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.228276 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdtx9\" (UniqueName: \"kubernetes.io/projected/ea712a68-bdb0-44ac-b949-fa324d02ede3-kube-api-access-sdtx9\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.228842 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.228749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-trusted-ca-bundle\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.228947 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.228892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-oauth-config\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.229039 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.229022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-serving-cert\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.229099 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.229085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-oauth-serving-cert\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330361 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-oauth-serving-cert\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330531 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330471 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-config\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330531 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-service-ca\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330649 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdtx9\" (UniqueName: \"kubernetes.io/projected/ea712a68-bdb0-44ac-b949-fa324d02ede3-kube-api-access-sdtx9\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330649 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-trusted-ca-bundle\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330649 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-oauth-config\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.330812 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.330670 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-serving-cert\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.331220 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.331184 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-oauth-serving-cert\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.331502 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.331478 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-service-ca\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.331603 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.331526 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-config\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.331786 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.331762 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-trusted-ca-bundle\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.333720 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.333686 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-serving-cert\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.334240 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.334197 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-oauth-config\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.339266 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.339247 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdtx9\" (UniqueName: \"kubernetes.io/projected/ea712a68-bdb0-44ac-b949-fa324d02ede3-kube-api-access-sdtx9\") pod \"console-6cf6bf6968-l4jrk\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.415275 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.415128 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rvpd8\"" Jul 22 18:44:57.424018 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.423944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mhpbl" Jul 22 18:44:57.443000 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.442978 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:44:57.900319 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.900245 2566 patch_prober.go:28] interesting pod/image-registry-785cd8c857-vhdvr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 22 18:44:57.900509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:57.900318 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" podUID="0c73ba3b-7767-41d8-9900-be044340308f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 18:44:58.550229 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.550160 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cf6bf6968-l4jrk"] Jul 22 18:44:58.554426 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:58.554390 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea712a68_bdb0_44ac_b949_fa324d02ede3.slice/crio-0157866a7bd90a2c250192b6171e089367522ff149631cb4e67a6b9dd7d84db9 WatchSource:0}: Error finding container 0157866a7bd90a2c250192b6171e089367522ff149631cb4e67a6b9dd7d84db9: Status 404 returned error can't find the container with id 0157866a7bd90a2c250192b6171e089367522ff149631cb4e67a6b9dd7d84db9 Jul 22 18:44:58.572966 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.572751 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mhpbl"] Jul 22 18:44:58.575653 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:44:58.575622 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfd6db7_5394_4b19_b9d9_8ecc9ce07c2c.slice/crio-fdf80668837146fb0be506e2df3c5a70e31de178a27ab8d563e707dd160cad59 WatchSource:0}: Error finding container fdf80668837146fb0be506e2df3c5a70e31de178a27ab8d563e707dd160cad59: Status 404 returned error can't find the container with id fdf80668837146fb0be506e2df3c5a70e31de178a27ab8d563e707dd160cad59 Jul 22 18:44:58.942110 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.942069 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccf648c-8f6pb" event={"ID":"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b","Type":"ContainerStarted","Data":"2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58"} Jul 22 18:44:58.943930 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.943862 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhpbl" event={"ID":"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c","Type":"ContainerStarted","Data":"fdf80668837146fb0be506e2df3c5a70e31de178a27ab8d563e707dd160cad59"} Jul 22 18:44:58.945923 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.945898 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf6bf6968-l4jrk" event={"ID":"ea712a68-bdb0-44ac-b949-fa324d02ede3","Type":"ContainerStarted","Data":"450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9"} Jul 22 18:44:58.946026 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.945941 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf6bf6968-l4jrk" event={"ID":"ea712a68-bdb0-44ac-b949-fa324d02ede3","Type":"ContainerStarted","Data":"0157866a7bd90a2c250192b6171e089367522ff149631cb4e67a6b9dd7d84db9"} Jul 22 18:44:58.960884 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.960829 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c8ccf648c-8f6pb" podStartSLOduration=1.8751011709999998 podStartE2EDuration="4.960796994s" totalImagesPullingTime="3.085695823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:58.959292324 +0000 UTC m=+307.477103271" watchObservedRunningTime="2026-07-22 18:44:58.960796994 +0000 UTC m=+307.478607905" Jul 22 18:44:58.975723 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:44:58.975682 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cf6bf6968-l4jrk" podStartSLOduration=1.975668706 podStartE2EDuration="1.975668706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:44:58.974514371 +0000 UTC m=+307.492325287" watchObservedRunningTime="2026-07-22 18:44:58.975668706 +0000 UTC m=+307.493479684" Jul 22 18:45:01.037789 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:01.037747 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:45:05.047075 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.047036 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:45:05.047549 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.047098 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:45:05.052984 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.052962 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:45:05.979585 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.979300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-tx8r5" event={"ID":"9460965c-3154-4b38-bfd3-dfee19bd240d","Type":"ContainerStarted","Data":"2385fd6cb8b01c8225c1fab8da76dd402d3a8cc3dc86abf829f118f0746f5e0a"} Jul 22 18:45:05.980236 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.979619 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:45:05.985282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.985256 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-tx8r5" Jul 22 18:45:05.987521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.986856 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:45:05.997326 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:05.997135 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-tx8r5" podStartSLOduration=1.6227817359999999 podStartE2EDuration="18.997115675s" totalImagesPullingTime="17.374333939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:44:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:05.996270414 +0000 UTC m=+314.514081329" watchObservedRunningTime="2026-07-22 18:45:05.997115675 +0000 UTC m=+314.514926592" Jul 22 18:45:07.444039 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:07.443982 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:45:07.444957 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:07.444289 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:45:07.450959 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:07.450901 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:45:07.899686 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:07.899655 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:45:07.993530 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:07.993498 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:45:08.048859 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:08.048420 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8ccf648c-8f6pb"] Jul 22 18:45:09.225129 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.225078 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rsctn"] Jul 22 18:45:09.383139 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.382811 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.389130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.388657 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 18:45:09.389130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.388703 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 18:45:09.389130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.388976 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 18:45:09.389130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.389017 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bns8p\"" Jul 22 18:45:09.389454 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.389254 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.435724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-textfile\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.435775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-sys\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.435836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.435874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-tls\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.435938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.435969 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-wtmp\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.436002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf4vx\" (UniqueName: \"kubernetes.io/projected/35e30752-2472-4f6e-b4a0-111a17f5a1a5-kube-api-access-vf4vx\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436542 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.436097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-root\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.436542 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.436192 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35e30752-2472-4f6e-b4a0-111a17f5a1a5-metrics-client-ca\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536695 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536695 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-wtmp\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf4vx\" (UniqueName: \"kubernetes.io/projected/35e30752-2472-4f6e-b4a0-111a17f5a1a5-kube-api-access-vf4vx\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536726 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-root\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35e30752-2472-4f6e-b4a0-111a17f5a1a5-metrics-client-ca\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-textfile\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536806 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-sys\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536898 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-tls\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.536951 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536928 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-root\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.537346 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.536990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-sys\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.537346 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.537220 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-wtmp\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.537653 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.537544 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35e30752-2472-4f6e-b4a0-111a17f5a1a5-metrics-client-ca\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.537844 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.537672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-accelerators-collector-config\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.537844 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.537797 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-textfile\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.540922 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.540899 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-tls\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.541998 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.541974 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35e30752-2472-4f6e-b4a0-111a17f5a1a5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.552492 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.552446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf4vx\" (UniqueName: \"kubernetes.io/projected/35e30752-2472-4f6e-b4a0-111a17f5a1a5-kube-api-access-vf4vx\") pod \"node-exporter-rsctn\" (UID: \"35e30752-2472-4f6e-b4a0-111a17f5a1a5\") " pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:09.698336 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:09.698295 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rsctn" Jul 22 18:45:10.253905 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.252604 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 18:45:10.394596 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.394562 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 18:45:10.394767 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.394720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.397220 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.397180 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 22 18:45:10.397220 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.397213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 22 18:45:10.397435 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.397336 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 22 18:45:10.397677 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.397652 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 22 18:45:10.397879 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.397863 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 22 18:45:10.398119 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.397978 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 22 18:45:10.398119 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.398087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 22 18:45:10.398459 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.398437 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jc87j\"" Jul 22 18:45:10.398648 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.398632 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 22 18:45:10.398897 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.398872 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 22 18:45:10.445335 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445309 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fc467fd1-fc00-489d-bc49-85e800af651c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445457 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-web-config\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445457 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-config-volume\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445457 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc467fd1-fc00-489d-bc49-85e800af651c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445457 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445447 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445651 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fc467fd1-fc00-489d-bc49-85e800af651c-config-out\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445651 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fc467fd1-fc00-489d-bc49-85e800af651c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445651 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445624 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mrjq\" (UniqueName: \"kubernetes.io/projected/fc467fd1-fc00-489d-bc49-85e800af651c-kube-api-access-8mrjq\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445670 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc467fd1-fc00-489d-bc49-85e800af651c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445698 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.445795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.445791 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546623 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546548 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fc467fd1-fc00-489d-bc49-85e800af651c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546623 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mrjq\" (UniqueName: \"kubernetes.io/projected/fc467fd1-fc00-489d-bc49-85e800af651c-kube-api-access-8mrjq\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546639 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc467fd1-fc00-489d-bc49-85e800af651c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546801 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fc467fd1-fc00-489d-bc49-85e800af651c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.546871 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-web-config\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.547214 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546889 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-config-volume\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.547214 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546919 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc467fd1-fc00-489d-bc49-85e800af651c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.547214 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.547214 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.546988 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fc467fd1-fc00-489d-bc49-85e800af651c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.547214 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.547013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fc467fd1-fc00-489d-bc49-85e800af651c-config-out\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.548018 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.547870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc467fd1-fc00-489d-bc49-85e800af651c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.548543 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.548476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc467fd1-fc00-489d-bc49-85e800af651c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.550498 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.550453 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.551394 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.551351 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-web-config\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.551504 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.551468 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fc467fd1-fc00-489d-bc49-85e800af651c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.551564 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.551516 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.551899 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.551881 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.552872 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.552319 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fc467fd1-fc00-489d-bc49-85e800af651c-config-out\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.552872 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.552594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.552872 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.552664 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.553885 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.553861 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fc467fd1-fc00-489d-bc49-85e800af651c-config-volume\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.554914 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.554793 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mrjq\" (UniqueName: \"kubernetes.io/projected/fc467fd1-fc00-489d-bc49-85e800af651c-kube-api-access-8mrjq\") pod \"alertmanager-main-0\" (UID: \"fc467fd1-fc00-489d-bc49-85e800af651c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:10.708052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:10.708010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 18:45:11.660044 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:45:11.660003 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35e30752_2472_4f6e_b4a0_111a17f5a1a5.slice/crio-2fef8212c73cc2809e8c4494f7be8bd05ee5697bc8fadac6f8e65a9faee9d225 WatchSource:0}: Error finding container 2fef8212c73cc2809e8c4494f7be8bd05ee5697bc8fadac6f8e65a9faee9d225: Status 404 returned error can't find the container with id 2fef8212c73cc2809e8c4494f7be8bd05ee5697bc8fadac6f8e65a9faee9d225 Jul 22 18:45:11.846557 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:11.846480 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 18:45:12.007423 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.007339 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rsctn" event={"ID":"35e30752-2472-4f6e-b4a0-111a17f5a1a5","Type":"ContainerStarted","Data":"2fef8212c73cc2809e8c4494f7be8bd05ee5697bc8fadac6f8e65a9faee9d225"} Jul 22 18:45:12.013124 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:45:12.013089 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc467fd1_fc00_489d_bc49_85e800af651c.slice/crio-99a7e1483b816d8c2052dc5d1e8d74a50439bdd1069aacc8732ab92ee111688b WatchSource:0}: Error finding container 99a7e1483b816d8c2052dc5d1e8d74a50439bdd1069aacc8732ab92ee111688b: Status 404 returned error can't find the container with id 99a7e1483b816d8c2052dc5d1e8d74a50439bdd1069aacc8732ab92ee111688b Jul 22 18:45:12.169064 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.169034 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-d9888b94c-cm6sb"] Jul 22 18:45:12.189740 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.189700 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d9888b94c-cm6sb"] Jul 22 18:45:12.189948 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.189853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.192234 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.192644 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e0bu3gjm0des9\"" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.192860 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qxbrv\"" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.193039 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.193067 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.193220 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 22 18:45:12.193850 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.193255 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 22 18:45:12.265376 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265376 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265344 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-tls\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265642 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265642 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265433 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265642 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-grpc-tls\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265642 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265566 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/595ee520-ff95-4c36-998c-ddd7099c4049-metrics-client-ca\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265642 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.265932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.265657 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6gp4\" (UniqueName: \"kubernetes.io/projected/595ee520-ff95-4c36-998c-ddd7099c4049-kube-api-access-g6gp4\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.366584 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-tls\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-grpc-tls\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/595ee520-ff95-4c36-998c-ddd7099c4049-metrics-client-ca\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6gp4\" (UniqueName: \"kubernetes.io/projected/595ee520-ff95-4c36-998c-ddd7099c4049-kube-api-access-g6gp4\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.367054 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.366905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.370785 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.370713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/595ee520-ff95-4c36-998c-ddd7099c4049-metrics-client-ca\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.370785 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.370747 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.371452 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.371399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.373333 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.373307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.374264 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.374244 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-grpc-tls\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.375602 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.375578 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-tls\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.375699 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.375605 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/595ee520-ff95-4c36-998c-ddd7099c4049-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.381208 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.381173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6gp4\" (UniqueName: \"kubernetes.io/projected/595ee520-ff95-4c36-998c-ddd7099c4049-kube-api-access-g6gp4\") pod \"thanos-querier-d9888b94c-cm6sb\" (UID: \"595ee520-ff95-4c36-998c-ddd7099c4049\") " pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.505804 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.505774 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:12.917477 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:12.917235 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" podUID="0c73ba3b-7767-41d8-9900-be044340308f" containerName="registry" containerID="cri-o://00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9" gracePeriod=30 Jul 22 18:45:13.013913 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.013874 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhpbl" event={"ID":"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c","Type":"ContainerStarted","Data":"92a2770fea3f987ac18633a5778ac5e164ea2afa3d120729bc215bc9e84b0fc1"} Jul 22 18:45:13.015301 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.015272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"99a7e1483b816d8c2052dc5d1e8d74a50439bdd1069aacc8732ab92ee111688b"} Jul 22 18:45:13.031183 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.031153 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d9888b94c-cm6sb"] Jul 22 18:45:13.035291 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:45:13.035257 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595ee520_ff95_4c36_998c_ddd7099c4049.slice/crio-2371f4f63ca7f4947417b716545fb5295395bf2372d22574a0c27271e42194e9 WatchSource:0}: Error finding container 2371f4f63ca7f4947417b716545fb5295395bf2372d22574a0c27271e42194e9: Status 404 returned error can't find the container with id 2371f4f63ca7f4947417b716545fb5295395bf2372d22574a0c27271e42194e9 Jul 22 18:45:13.242794 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.242771 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275674 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-installation-pull-secrets\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275716 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8wlm\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-kube-api-access-z8wlm\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275772 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-bound-sa-token\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275804 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275886 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-image-registry-private-configuration\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275940 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-registry-certificates\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.275981 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-trusted-ca\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.277811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.276011 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c73ba3b-7767-41d8-9900-be044340308f-ca-trust-extracted\") pod \"0c73ba3b-7767-41d8-9900-be044340308f\" (UID: \"0c73ba3b-7767-41d8-9900-be044340308f\") " Jul 22 18:45:13.279908 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.279880 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:13.280081 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.280051 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:13.283634 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.283585 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:45:13.286021 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.285982 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:45:13.286021 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.286000 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-kube-api-access-z8wlm" (OuterVolumeSpecName: "kube-api-access-z8wlm") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "kube-api-access-z8wlm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:45:13.286493 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.286472 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:45:13.287043 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.287003 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:45:13.292472 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.292447 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c73ba3b-7767-41d8-9900-be044340308f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0c73ba3b-7767-41d8-9900-be044340308f" (UID: "0c73ba3b-7767-41d8-9900-be044340308f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:45:13.377097 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377056 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-installation-pull-secrets\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377097 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377095 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8wlm\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-kube-api-access-z8wlm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377110 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-bound-sa-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377125 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0c73ba3b-7767-41d8-9900-be044340308f-registry-tls\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377141 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0c73ba3b-7767-41d8-9900-be044340308f-image-registry-private-configuration\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377156 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-registry-certificates\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377170 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c73ba3b-7767-41d8-9900-be044340308f-trusted-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.377382 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.377186 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0c73ba3b-7767-41d8-9900-be044340308f-ca-trust-extracted\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:13.488493 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.488311 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6c4657d6ff-dlc9q"] Jul 22 18:45:13.488851 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.488810 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c73ba3b-7767-41d8-9900-be044340308f" containerName="registry" Jul 22 18:45:13.488851 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.488848 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c73ba3b-7767-41d8-9900-be044340308f" containerName="registry" Jul 22 18:45:13.489010 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.488925 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c73ba3b-7767-41d8-9900-be044340308f" containerName="registry" Jul 22 18:45:13.503942 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.503913 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c4657d6ff-dlc9q"] Jul 22 18:45:13.504100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.504031 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.507881 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.507118 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 22 18:45:13.507881 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.507334 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 18:45:13.507881 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.507382 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 22 18:45:13.507881 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.507561 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-94dj5\"" Jul 22 18:45:13.507881 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.507728 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 22 18:45:13.509192 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.509172 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-19jpvrof82c9r\"" Jul 22 18:45:13.579371 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450400fd-9b4e-4737-9b00-40de387bd711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.579527 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579391 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-client-ca-bundle\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.579527 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-secret-metrics-server-tls\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.579527 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/450400fd-9b4e-4737-9b00-40de387bd711-audit-log\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.579689 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579546 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/450400fd-9b4e-4737-9b00-40de387bd711-metrics-server-audit-profiles\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.579689 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-secret-metrics-server-client-certs\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.579782 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.579700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grzgh\" (UniqueName: \"kubernetes.io/projected/450400fd-9b4e-4737-9b00-40de387bd711-kube-api-access-grzgh\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.680765 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.680728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450400fd-9b4e-4737-9b00-40de387bd711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.680917 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.680786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-client-ca-bundle\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.680917 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.680834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-secret-metrics-server-tls\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.680917 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.680878 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/450400fd-9b4e-4737-9b00-40de387bd711-audit-log\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.681070 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.680956 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/450400fd-9b4e-4737-9b00-40de387bd711-metrics-server-audit-profiles\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.681070 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.680996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-secret-metrics-server-client-certs\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.681070 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.681023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grzgh\" (UniqueName: \"kubernetes.io/projected/450400fd-9b4e-4737-9b00-40de387bd711-kube-api-access-grzgh\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.681995 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.681735 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/450400fd-9b4e-4737-9b00-40de387bd711-audit-log\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.684006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.683388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/450400fd-9b4e-4737-9b00-40de387bd711-metrics-server-audit-profiles\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.684006 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.683960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/450400fd-9b4e-4737-9b00-40de387bd711-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.684764 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.684376 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-secret-metrics-server-tls\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.689986 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.689942 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-client-ca-bundle\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.690188 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.690152 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/450400fd-9b4e-4737-9b00-40de387bd711-secret-metrics-server-client-certs\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.693089 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.693015 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grzgh\" (UniqueName: \"kubernetes.io/projected/450400fd-9b4e-4737-9b00-40de387bd711-kube-api-access-grzgh\") pod \"metrics-server-6c4657d6ff-dlc9q\" (UID: \"450400fd-9b4e-4737-9b00-40de387bd711\") " pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.823594 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.823259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:13.956628 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.955193 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb"] Jul 22 18:45:13.983029 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.982250 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb"] Jul 22 18:45:13.983029 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.982381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:13.988357 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.986994 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 22 18:45:13.992125 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:13.991993 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fr4gb\"" Jul 22 18:45:14.024991 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.022023 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"2371f4f63ca7f4947417b716545fb5295395bf2372d22574a0c27271e42194e9"} Jul 22 18:45:14.024991 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.024096 2566 generic.go:356] "Generic (PLEG): container finished" podID="0c73ba3b-7767-41d8-9900-be044340308f" containerID="00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9" exitCode=0 Jul 22 18:45:14.024991 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.024165 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" Jul 22 18:45:14.024991 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.024186 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" event={"ID":"0c73ba3b-7767-41d8-9900-be044340308f","Type":"ContainerDied","Data":"00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9"} Jul 22 18:45:14.024991 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.024246 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-785cd8c857-vhdvr" event={"ID":"0c73ba3b-7767-41d8-9900-be044340308f","Type":"ContainerDied","Data":"baee53dd8b35ae9d55abafce192ce13cbb74c5b9e67d84ab17b5474b34d1efa1"} Jul 22 18:45:14.024991 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.024280 2566 scope.go:117] "RemoveContainer" containerID="00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9" Jul 22 18:45:14.029895 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.029293 2566 generic.go:356] "Generic (PLEG): container finished" podID="35e30752-2472-4f6e-b4a0-111a17f5a1a5" containerID="0896166cf62912d86a819abb009cb3058c4ed0437c2919088a8bc35d7e6d41c1" exitCode=0 Jul 22 18:45:14.029895 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.029338 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rsctn" event={"ID":"35e30752-2472-4f6e-b4a0-111a17f5a1a5","Type":"ContainerDied","Data":"0896166cf62912d86a819abb009cb3058c4ed0437c2919088a8bc35d7e6d41c1"} Jul 22 18:45:14.040780 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.040759 2566 scope.go:117] "RemoveContainer" containerID="00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9" Jul 22 18:45:14.041269 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:45:14.041238 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9\": container with ID starting with 00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9 not found: ID does not exist" containerID="00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9" Jul 22 18:45:14.041392 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.041296 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9"} err="failed to get container status \"00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9\": rpc error: code = NotFound desc = could not find container \"00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9\": container with ID starting with 00c9e986ee481e91fe8b7f5196af593a989342cc318d7bbde27690356b4a6dd9 not found: ID does not exist" Jul 22 18:45:14.054395 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.054334 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6c4657d6ff-dlc9q"] Jul 22 18:45:14.059480 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:45:14.059449 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450400fd_9b4e_4737_9b00_40de387bd711.slice/crio-a9ebe9f6cd04a340e633bf0a32ba7c51b8c6020a849f4e5873ec01ba6066a438 WatchSource:0}: Error finding container a9ebe9f6cd04a340e633bf0a32ba7c51b8c6020a849f4e5873ec01ba6066a438: Status 404 returned error can't find the container with id a9ebe9f6cd04a340e633bf0a32ba7c51b8c6020a849f4e5873ec01ba6066a438 Jul 22 18:45:14.068385 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.068337 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-785cd8c857-vhdvr"] Jul 22 18:45:14.075964 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.075938 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-785cd8c857-vhdvr"] Jul 22 18:45:14.087449 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.087375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4664\" (UniqueName: \"kubernetes.io/projected/2fd46e12-ba52-4558-961c-6e97a82496ec-kube-api-access-h4664\") pod \"monitoring-plugin-7b7b7bc8bf-k4hfb\" (UID: \"2fd46e12-ba52-4558-961c-6e97a82496ec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.088276 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.087709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fd46e12-ba52-4558-961c-6e97a82496ec-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k4hfb\" (UID: \"2fd46e12-ba52-4558-961c-6e97a82496ec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.189558 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.188843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fd46e12-ba52-4558-961c-6e97a82496ec-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k4hfb\" (UID: \"2fd46e12-ba52-4558-961c-6e97a82496ec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.189558 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.188931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4664\" (UniqueName: \"kubernetes.io/projected/2fd46e12-ba52-4558-961c-6e97a82496ec-kube-api-access-h4664\") pod \"monitoring-plugin-7b7b7bc8bf-k4hfb\" (UID: \"2fd46e12-ba52-4558-961c-6e97a82496ec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.192668 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.192644 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fd46e12-ba52-4558-961c-6e97a82496ec-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k4hfb\" (UID: \"2fd46e12-ba52-4558-961c-6e97a82496ec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.198341 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.198317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4664\" (UniqueName: \"kubernetes.io/projected/2fd46e12-ba52-4558-961c-6e97a82496ec-kube-api-access-h4664\") pod \"monitoring-plugin-7b7b7bc8bf-k4hfb\" (UID: \"2fd46e12-ba52-4558-961c-6e97a82496ec\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.302265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.302231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:14.319048 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.319003 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d64b4fb88-xzndf"] Jul 22 18:45:14.345105 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.343762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d64b4fb88-xzndf"] Jul 22 18:45:14.345105 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.343904 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391309 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391205 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-oauth-serving-cert\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391309 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391283 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-oauth-config\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391458 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-serving-cert\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391458 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391376 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-service-ca\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391458 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wkvq\" (UniqueName: \"kubernetes.io/projected/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-kube-api-access-4wkvq\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391603 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-config\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.391603 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.391511 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-trusted-ca-bundle\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492254 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-serving-cert\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492254 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-service-ca\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492498 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wkvq\" (UniqueName: \"kubernetes.io/projected/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-kube-api-access-4wkvq\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492498 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-config\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492498 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492399 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-trusted-ca-bundle\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492498 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-oauth-serving-cert\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.492692 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.492512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-oauth-config\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.495980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.494315 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-service-ca\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.495980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.495289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-config\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.495980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.495911 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-oauth-serving-cert\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.497134 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.496999 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-serving-cert\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.497134 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.497074 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-trusted-ca-bundle\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.497903 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.497849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-oauth-config\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.500363 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.500249 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb"] Jul 22 18:45:14.503672 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.503617 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wkvq\" (UniqueName: \"kubernetes.io/projected/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-kube-api-access-4wkvq\") pod \"console-6d64b4fb88-xzndf\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.657765 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:14.657724 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:14.927946 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:45:14.927866 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd46e12_ba52_4558_961c_6e97a82496ec.slice/crio-41b9c3d5db208af068d2b2fd313b2287fdccb94483acd0f34322e1b0193b7375 WatchSource:0}: Error finding container 41b9c3d5db208af068d2b2fd313b2287fdccb94483acd0f34322e1b0193b7375: Status 404 returned error can't find the container with id 41b9c3d5db208af068d2b2fd313b2287fdccb94483acd0f34322e1b0193b7375 Jul 22 18:45:15.055657 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:15.055585 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" event={"ID":"2fd46e12-ba52-4558-961c-6e97a82496ec","Type":"ContainerStarted","Data":"41b9c3d5db208af068d2b2fd313b2287fdccb94483acd0f34322e1b0193b7375"} Jul 22 18:45:15.060319 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:15.060270 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rsctn" event={"ID":"35e30752-2472-4f6e-b4a0-111a17f5a1a5","Type":"ContainerStarted","Data":"63feea0f1f82b23139f6cb3284c9158cba8e2713fcb563ea14ea4d72ac7a25ab"} Jul 22 18:45:15.060546 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:15.060526 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rsctn" event={"ID":"35e30752-2472-4f6e-b4a0-111a17f5a1a5","Type":"ContainerStarted","Data":"77540a85a6bab99b2330e76e7f25b039a3e9114413f6cd6d9f74d6ac111217ff"} Jul 22 18:45:15.063409 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:15.063385 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" event={"ID":"450400fd-9b4e-4737-9b00-40de387bd711","Type":"ContainerStarted","Data":"a9ebe9f6cd04a340e633bf0a32ba7c51b8c6020a849f4e5873ec01ba6066a438"} Jul 22 18:45:15.082199 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:15.080970 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rsctn" podStartSLOduration=4.405207048 podStartE2EDuration="6.080950337s" totalImagesPullingTime="1.578125806s" totalInitContainerRuntime="97.617483ms" isStatefulPod=true podCreationTimestamp="2026-07-22 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:15.079352398 +0000 UTC m=+323.597163335" watchObservedRunningTime="2026-07-22 18:45:15.080950337 +0000 UTC m=+323.598761252" Jul 22 18:45:15.109569 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:15.109494 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d64b4fb88-xzndf"] Jul 22 18:45:15.218941 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:45:15.218867 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce38c2c_0e5d_4f9b_b4f9_2e03b2b523ef.slice/crio-636cd88844c8afadce0734aab8add32813695fb526f2f7a4932f6b2c31f8cc5e WatchSource:0}: Error finding container 636cd88844c8afadce0734aab8add32813695fb526f2f7a4932f6b2c31f8cc5e: Status 404 returned error can't find the container with id 636cd88844c8afadce0734aab8add32813695fb526f2f7a4932f6b2c31f8cc5e Jul 22 18:45:16.043590 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:16.043557 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c73ba3b-7767-41d8-9900-be044340308f" path="/var/lib/kubelet/pods/0c73ba3b-7767-41d8-9900-be044340308f/volumes" Jul 22 18:45:16.071889 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:16.071850 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"30d73d890a6732ac8bdd8e427a8cd22305a2a9c9785ef9c6a6e1aac3e75234bb"} Jul 22 18:45:16.073744 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:16.073709 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64b4fb88-xzndf" event={"ID":"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef","Type":"ContainerStarted","Data":"ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a"} Jul 22 18:45:16.073930 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:16.073766 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64b4fb88-xzndf" event={"ID":"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef","Type":"ContainerStarted","Data":"636cd88844c8afadce0734aab8add32813695fb526f2f7a4932f6b2c31f8cc5e"} Jul 22 18:45:16.113472 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:16.113052 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d64b4fb88-xzndf" podStartSLOduration=2.113029921 podStartE2EDuration="2.113029921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:16.112789043 +0000 UTC m=+324.630599957" watchObservedRunningTime="2026-07-22 18:45:16.113029921 +0000 UTC m=+324.630840814" Jul 22 18:45:17.079623 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:17.079590 2566 generic.go:356] "Generic (PLEG): container finished" podID="fc467fd1-fc00-489d-bc49-85e800af651c" containerID="30d73d890a6732ac8bdd8e427a8cd22305a2a9c9785ef9c6a6e1aac3e75234bb" exitCode=0 Jul 22 18:45:17.080207 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:17.080101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerDied","Data":"30d73d890a6732ac8bdd8e427a8cd22305a2a9c9785ef9c6a6e1aac3e75234bb"} Jul 22 18:45:19.092236 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:19.092202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"fc59b8f552a1dee4cd366d37a0b52fe457ad85be842ed254efbe44b05004f07c"} Jul 22 18:45:20.097568 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:20.097528 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhpbl" event={"ID":"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c","Type":"ContainerStarted","Data":"54610d486d98d943b55c5ba89248bf2ca43758ff04b006d7c60ef22778fa843b"} Jul 22 18:45:20.099241 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:20.099210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"9e053d1f0820ff2719c60c09c5354ff22bf610d2f0f1fea05c13e50be3c71e5c"} Jul 22 18:45:22.110271 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.110181 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" event={"ID":"2fd46e12-ba52-4558-961c-6e97a82496ec","Type":"ContainerStarted","Data":"7a7486b2790e6eff0e707baa400d77773e13aeaea342e9636ce1e2f31b1941b1"} Jul 22 18:45:22.110715 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.110613 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:22.113809 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.113778 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"c345747c340a9f946f4680b2b89217adda878f50aa9e584cae2617205eb27998"} Jul 22 18:45:22.113967 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.113882 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"a3ba48f2ef48ae871131226a43bdbdf4bfd14acf95093e7b79ba3bd34f975323"} Jul 22 18:45:22.113967 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.113930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"3f8b0d860ff4dad40aeceaa8d4762e0b7c6fc8bb01b461cfe0c2f4301d2e392a"} Jul 22 18:45:22.114078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.113969 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"4d51ca19c37fe73c6ec9ca8f68123d5b9d52c6909f4f23f05a7891ab90c51f49"} Jul 22 18:45:22.114078 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.114007 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"1206be52f6165ee19b879234f743a5414ebbd85ef40d8dda8cb4393f2ee43c43"} Jul 22 18:45:22.116109 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.116085 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" Jul 22 18:45:22.116504 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.116479 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"7c97d28a30e3e79fb919096054aa7a2aaea74c5a2e9b5a896806d54874b33f02"} Jul 22 18:45:22.117804 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.117781 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" event={"ID":"450400fd-9b4e-4737-9b00-40de387bd711","Type":"ContainerStarted","Data":"deff49c7d0ae978a39c96f0e7ffdcb47db70d59df87de59aeeea08a5d7565a76"} Jul 22 18:45:22.124613 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.124574 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k4hfb" podStartSLOduration=2.998421952 podStartE2EDuration="9.124561758s" totalImagesPullingTime="6.126139806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:22.123758857 +0000 UTC m=+330.641569783" watchObservedRunningTime="2026-07-22 18:45:22.124561758 +0000 UTC m=+330.642372692" Jul 22 18:45:22.153398 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:22.153327 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" podStartSLOduration=2.157577707 podStartE2EDuration="9.153306289s" totalImagesPullingTime="6.995728582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:45:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:22.140106154 +0000 UTC m=+330.657917070" watchObservedRunningTime="2026-07-22 18:45:22.153306289 +0000 UTC m=+330.671117183" Jul 22 18:45:23.126580 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:23.126547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fc467fd1-fc00-489d-bc49-85e800af651c","Type":"ContainerStarted","Data":"7256867fdbfb6d90a2ac2fc47f4788aad3fd7842fe649fb99750112f9a52e3c7"} Jul 22 18:45:23.162726 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:23.162671 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.82801885 podStartE2EDuration="13.162649423s" totalImagesPullingTime="8.315056744s" totalInitContainerRuntime="19.573829ms" isStatefulPod=false podCreationTimestamp="2026-07-22 18:45:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:23.161327222 +0000 UTC m=+331.679138138" watchObservedRunningTime="2026-07-22 18:45:23.162649423 +0000 UTC m=+331.680460335" Jul 22 18:45:24.134359 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.134318 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"38071d5989b0801faca12bbf6030461800307db175242ffd3b5bb47b91d372bb"} Jul 22 18:45:24.134837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.134413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"6c488fd74a4eee8a1d4ce0b765870223cc49b904f0735813729666be9524c274"} Jul 22 18:45:24.134837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.134443 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" event={"ID":"595ee520-ff95-4c36-998c-ddd7099c4049","Type":"ContainerStarted","Data":"29fbd03f3615f26d9f9ea7350344bfb30e2aed9bf72ae4513a92d382c06bc4bd"} Jul 22 18:45:24.134837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.134644 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:24.141155 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.141129 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" Jul 22 18:45:24.158616 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.158563 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d9888b94c-cm6sb" podStartSLOduration=4.551820613 podStartE2EDuration="12.158541125s" totalImagesPullingTime="7.606720512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:45:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:24.156299132 +0000 UTC m=+332.674110072" watchObservedRunningTime="2026-07-22 18:45:24.158541125 +0000 UTC m=+332.676352042" Jul 22 18:45:24.659031 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.658990 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:24.659220 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.659080 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:24.665131 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:24.665104 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:25.144094 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:25.144061 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:45:25.193286 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:25.193239 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cf6bf6968-l4jrk"] Jul 22 18:45:26.145502 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:26.145463 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mhpbl" event={"ID":"3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c","Type":"ContainerStarted","Data":"2d604192a6a0bd7c19d070e405275dc79bb94565e1b03f3a61d38de652ab6fb0"} Jul 22 18:45:26.162644 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:26.162591 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mhpbl" podStartSLOduration=34.087222095 podStartE2EDuration="1m1.162577517s" totalImagesPullingTime="27.075355422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 18:44:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:45:26.161293884 +0000 UTC m=+334.679104798" watchObservedRunningTime="2026-07-22 18:45:26.162577517 +0000 UTC m=+334.680388430" Jul 22 18:45:33.076978 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.076940 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c8ccf648c-8f6pb" podUID="ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" containerName="console" containerID="cri-o://2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58" gracePeriod=15 Jul 22 18:45:33.367549 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.367514 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:45:33.479728 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.479699 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-oauth-config\") pod \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " Jul 22 18:45:33.479971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.479760 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-oauth-serving-cert\") pod \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " Jul 22 18:45:33.479971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.479789 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmdq4\" (UniqueName: \"kubernetes.io/projected/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-kube-api-access-pmdq4\") pod \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " Jul 22 18:45:33.479971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.479907 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-service-ca\") pod \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " Jul 22 18:45:33.479971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.479959 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-config\") pod \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " Jul 22 18:45:33.480177 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.479999 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-serving-cert\") pod \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\" (UID: \"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b\") " Jul 22 18:45:33.480225 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.480166 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" (UID: "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:33.480291 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.480263 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-service-ca" (OuterVolumeSpecName: "service-ca") pod "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" (UID: "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:33.480365 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.480342 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-config" (OuterVolumeSpecName: "console-config") pod "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" (UID: "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:33.480419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.480376 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-oauth-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:33.480419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.480396 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-service-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:33.482040 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.482015 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-kube-api-access-pmdq4" (OuterVolumeSpecName: "kube-api-access-pmdq4") pod "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" (UID: "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b"). InnerVolumeSpecName "kube-api-access-pmdq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:45:33.482150 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.482049 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" (UID: "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:45:33.482150 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.482072 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" (UID: "ea8a0f24-1d15-4c5d-95cd-6150abf66f8b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:45:33.580898 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.580868 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-oauth-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:33.580898 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.580895 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmdq4\" (UniqueName: \"kubernetes.io/projected/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-kube-api-access-pmdq4\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:33.580898 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.580906 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:33.581130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.580915 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b-console-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:33.823623 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.823577 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:33.823955 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:33.823659 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:34.184415 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.184385 2566 generic.go:356] "Generic (PLEG): container finished" podID="ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" containerID="2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58" exitCode=0 Jul 22 18:45:34.184866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.184426 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c8ccf648c-8f6pb" Jul 22 18:45:34.184866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.184426 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccf648c-8f6pb" event={"ID":"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b","Type":"ContainerDied","Data":"2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58"} Jul 22 18:45:34.184866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.184572 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c8ccf648c-8f6pb" event={"ID":"ea8a0f24-1d15-4c5d-95cd-6150abf66f8b","Type":"ContainerDied","Data":"faf16f92a833e99e1f21e5e6c9396eefbba6f2c62f4978cbf65aaa9974907d3a"} Jul 22 18:45:34.184866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.184609 2566 scope.go:117] "RemoveContainer" containerID="2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58" Jul 22 18:45:34.192929 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.192910 2566 scope.go:117] "RemoveContainer" containerID="2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58" Jul 22 18:45:34.193173 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:45:34.193149 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58\": container with ID starting with 2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58 not found: ID does not exist" containerID="2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58" Jul 22 18:45:34.193235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.193182 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58"} err="failed to get container status \"2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58\": rpc error: code = NotFound desc = could not find container \"2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58\": container with ID starting with 2a11e5ae560538c9b5a39b6fd5d955e8acc97a0c3db4768acce24449c2d6db58 not found: ID does not exist" Jul 22 18:45:34.201778 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.201757 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c8ccf648c-8f6pb"] Jul 22 18:45:34.206191 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:34.206171 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c8ccf648c-8f6pb"] Jul 22 18:45:36.042959 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:36.042924 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" path="/var/lib/kubelet/pods/ea8a0f24-1d15-4c5d-95cd-6150abf66f8b/volumes" Jul 22 18:45:40.209598 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:40.209568 2566 generic.go:356] "Generic (PLEG): container finished" podID="2722a962-6658-4f2a-a0ff-2bbfd3cec7e1" containerID="661c4d5651a239773e4136fd5467de18e82bbbb37dbcfe338705330d2ea57be5" exitCode=0 Jul 22 18:45:40.209971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:40.209616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" event={"ID":"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1","Type":"ContainerDied","Data":"661c4d5651a239773e4136fd5467de18e82bbbb37dbcfe338705330d2ea57be5"} Jul 22 18:45:40.209971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:40.209948 2566 scope.go:117] "RemoveContainer" containerID="661c4d5651a239773e4136fd5467de18e82bbbb37dbcfe338705330d2ea57be5" Jul 22 18:45:41.215052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:41.215019 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-jkzgp" event={"ID":"2722a962-6658-4f2a-a0ff-2bbfd3cec7e1","Type":"ContainerStarted","Data":"af60b9232a60fc9df9f36a075473f1e8ffbe61f26d34ef97bf7d5efda5446f95"} Jul 22 18:45:45.231626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.231593 2566 generic.go:356] "Generic (PLEG): container finished" podID="1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63" containerID="427ae76760c763e3cf44a7f91e7b34ba2e4ea06c507064de2831356ef29b1f21" exitCode=0 Jul 22 18:45:45.232035 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.231665 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" event={"ID":"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63","Type":"ContainerDied","Data":"427ae76760c763e3cf44a7f91e7b34ba2e4ea06c507064de2831356ef29b1f21"} Jul 22 18:45:45.232178 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.232159 2566 scope.go:117] "RemoveContainer" containerID="427ae76760c763e3cf44a7f91e7b34ba2e4ea06c507064de2831356ef29b1f21" Jul 22 18:45:45.233376 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.233357 2566 generic.go:356] "Generic (PLEG): container finished" podID="fbde1910-202e-40d4-87cc-fe1c3b781290" containerID="597641f091850fc0b56e727c7759513049a9d12e422c0077592ecb5fa0e03e16" exitCode=0 Jul 22 18:45:45.233467 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.233404 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-skfzm" event={"ID":"fbde1910-202e-40d4-87cc-fe1c3b781290","Type":"ContainerDied","Data":"597641f091850fc0b56e727c7759513049a9d12e422c0077592ecb5fa0e03e16"} Jul 22 18:45:45.233662 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.233650 2566 scope.go:117] "RemoveContainer" containerID="597641f091850fc0b56e727c7759513049a9d12e422c0077592ecb5fa0e03e16" Jul 22 18:45:45.964641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.964614 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b8f49c9b6-w6xm6_93d054bc-ebb9-4490-9fbc-cfe880af248f/router/0.log" Jul 22 18:45:45.990415 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:45.990391 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdkfp_71b46f3d-ad01-4915-91d3-03d2e769d731/serve-healthcheck-canary/0.log" Jul 22 18:45:46.238718 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:46.238638 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-kcvp9" event={"ID":"1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63","Type":"ContainerStarted","Data":"80f8bedf2736cd34ee5dbc298763ae4b331d80a715327a41db5b987faf84e66d"} Jul 22 18:45:46.240439 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:46.240415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-skfzm" event={"ID":"fbde1910-202e-40d4-87cc-fe1c3b781290","Type":"ContainerStarted","Data":"5e0f091e0a7af4431693d011e94d328ee2bcadca541b1d092f78d2b3b34d5809"} Jul 22 18:45:50.232086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.232038 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cf6bf6968-l4jrk" podUID="ea712a68-bdb0-44ac-b949-fa324d02ede3" containerName="console" containerID="cri-o://450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9" gracePeriod=15 Jul 22 18:45:50.500204 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.500139 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:45:50.531470 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531445 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-oauth-config\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.531636 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531483 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-trusted-ca-bundle\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.531636 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531562 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-service-ca\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.531636 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531596 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-serving-cert\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.531636 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531622 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-config\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.531887 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531669 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdtx9\" (UniqueName: \"kubernetes.io/projected/ea712a68-bdb0-44ac-b949-fa324d02ede3-kube-api-access-sdtx9\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.531887 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.531714 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-oauth-serving-cert\") pod \"ea712a68-bdb0-44ac-b949-fa324d02ede3\" (UID: \"ea712a68-bdb0-44ac-b949-fa324d02ede3\") " Jul 22 18:45:50.532328 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.532256 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-service-ca" (OuterVolumeSpecName: "service-ca") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:50.532328 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.532283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:50.532477 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.532416 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:50.532477 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.532421 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-config" (OuterVolumeSpecName: "console-config") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:45:50.534111 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.534083 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:45:50.534198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.534096 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea712a68-bdb0-44ac-b949-fa324d02ede3-kube-api-access-sdtx9" (OuterVolumeSpecName: "kube-api-access-sdtx9") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "kube-api-access-sdtx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:45:50.534198 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.534140 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ea712a68-bdb0-44ac-b949-fa324d02ede3" (UID: "ea712a68-bdb0-44ac-b949-fa324d02ede3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:45:50.632577 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632544 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-oauth-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:50.632577 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632572 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-trusted-ca-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:50.632577 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632582 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-service-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:50.632784 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632591 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:50.632784 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632600 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-console-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:50.632784 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632608 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdtx9\" (UniqueName: \"kubernetes.io/projected/ea712a68-bdb0-44ac-b949-fa324d02ede3-kube-api-access-sdtx9\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:50.632784 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:50.632617 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea712a68-bdb0-44ac-b949-fa324d02ede3-oauth-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:45:51.263665 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.263627 2566 generic.go:356] "Generic (PLEG): container finished" podID="ea712a68-bdb0-44ac-b949-fa324d02ede3" containerID="450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9" exitCode=0 Jul 22 18:45:51.264108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.263671 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cf6bf6968-l4jrk" Jul 22 18:45:51.264108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.263718 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf6bf6968-l4jrk" event={"ID":"ea712a68-bdb0-44ac-b949-fa324d02ede3","Type":"ContainerDied","Data":"450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9"} Jul 22 18:45:51.264108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.263772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cf6bf6968-l4jrk" event={"ID":"ea712a68-bdb0-44ac-b949-fa324d02ede3","Type":"ContainerDied","Data":"0157866a7bd90a2c250192b6171e089367522ff149631cb4e67a6b9dd7d84db9"} Jul 22 18:45:51.264108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.263794 2566 scope.go:117] "RemoveContainer" containerID="450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9" Jul 22 18:45:51.272235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.272221 2566 scope.go:117] "RemoveContainer" containerID="450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9" Jul 22 18:45:51.272466 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:45:51.272450 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9\": container with ID starting with 450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9 not found: ID does not exist" containerID="450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9" Jul 22 18:45:51.272504 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.272474 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9"} err="failed to get container status \"450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9\": rpc error: code = NotFound desc = could not find container \"450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9\": container with ID starting with 450f9033a039bcc2ffbf9a7571438e7c27324400c59260433ea023299ec33de9 not found: ID does not exist" Jul 22 18:45:51.283375 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.283352 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cf6bf6968-l4jrk"] Jul 22 18:45:51.286337 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:51.286311 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cf6bf6968-l4jrk"] Jul 22 18:45:52.043120 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:52.043091 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea712a68-bdb0-44ac-b949-fa324d02ede3" path="/var/lib/kubelet/pods/ea712a68-bdb0-44ac-b949-fa324d02ede3/volumes" Jul 22 18:45:53.828989 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:53.828961 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:45:53.832955 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:45:53.832933 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6c4657d6ff-dlc9q" Jul 22 18:46:05.867725 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:05.867684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:46:05.870051 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:05.870016 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf81b03f-0e6d-48dc-bafa-95e60c0c50a9-metrics-certs\") pod \"network-metrics-daemon-sjfmj\" (UID: \"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9\") " pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:46:06.141228 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:06.141138 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g2vpr\"" Jul 22 18:46:06.149187 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:06.149164 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjfmj" Jul 22 18:46:06.288743 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:06.288615 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sjfmj"] Jul 22 18:46:06.291487 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:46:06.291459 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf81b03f_0e6d_48dc_bafa_95e60c0c50a9.slice/crio-d9eca896586fd3020263920468069e6ce894ca04376d375f7c10318697e0496f WatchSource:0}: Error finding container d9eca896586fd3020263920468069e6ce894ca04376d375f7c10318697e0496f: Status 404 returned error can't find the container with id d9eca896586fd3020263920468069e6ce894ca04376d375f7c10318697e0496f Jul 22 18:46:06.331962 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:06.331927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjfmj" event={"ID":"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9","Type":"ContainerStarted","Data":"d9eca896586fd3020263920468069e6ce894ca04376d375f7c10318697e0496f"} Jul 22 18:46:08.342130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:08.342085 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjfmj" event={"ID":"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9","Type":"ContainerStarted","Data":"58edc920f84b91d55b92b7738488f0c94c79bbcc2ae2a533f8fc5c1e73605b4c"} Jul 22 18:46:08.342569 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:08.342162 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjfmj" event={"ID":"cf81b03f-0e6d-48dc-bafa-95e60c0c50a9","Type":"ContainerStarted","Data":"0cc704eede63c1d69cf33dba4b9b775efcab64e9ae8ecf35e77671684ac691ee"} Jul 22 18:46:08.366643 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:08.366592 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sjfmj" podStartSLOduration=375.374480451 podStartE2EDuration="6m16.366578544s" totalImagesPullingTime="992.098093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:39:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:46:08.364863198 +0000 UTC m=+376.882674113" watchObservedRunningTime="2026-07-22 18:46:08.366578544 +0000 UTC m=+376.884389458" Jul 22 18:46:33.741248 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741206 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf"] Jul 22 18:46:33.741734 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741715 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea712a68-bdb0-44ac-b949-fa324d02ede3" containerName="console" Jul 22 18:46:33.741790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741738 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea712a68-bdb0-44ac-b949-fa324d02ede3" containerName="console" Jul 22 18:46:33.741790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741776 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" containerName="console" Jul 22 18:46:33.741790 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741785 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" containerName="console" Jul 22 18:46:33.741914 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741880 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8a0f24-1d15-4c5d-95cd-6150abf66f8b" containerName="console" Jul 22 18:46:33.741914 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.741892 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea712a68-bdb0-44ac-b949-fa324d02ede3" containerName="console" Jul 22 18:46:33.745235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.745215 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.748518 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.748494 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 22 18:46:33.748644 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.748496 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 22 18:46:33.749215 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.749197 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 22 18:46:33.749627 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.749609 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 22 18:46:33.755125 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.755104 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-x47ph\"" Jul 22 18:46:33.756236 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.756218 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 22 18:46:33.756445 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.756429 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 22 18:46:33.771240 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.771218 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf"] Jul 22 18:46:33.801402 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801508 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-serving-certs-ca-bundle\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801508 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801433 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-secret-telemeter-client\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801508 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-metrics-client-ca\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801630 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-telemeter-client-tls\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801630 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-federate-client-tls\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801630 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801606 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6ssh\" (UniqueName: \"kubernetes.io/projected/5d6b1ef2-cacb-40f5-917d-814193a13f11-kube-api-access-j6ssh\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.801739 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.801641 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902380 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-serving-certs-ca-bundle\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902380 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902388 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-secret-telemeter-client\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-metrics-client-ca\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902428 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-telemeter-client-tls\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-federate-client-tls\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902491 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6ssh\" (UniqueName: \"kubernetes.io/projected/5d6b1ef2-cacb-40f5-917d-814193a13f11-kube-api-access-j6ssh\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902841 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902606 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.902841 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.902667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.903262 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.903238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-metrics-client-ca\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.903373 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.903238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-serving-certs-ca-bundle\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.903598 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.903575 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d6b1ef2-cacb-40f5-917d-814193a13f11-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.905273 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.905243 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-federate-client-tls\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.905353 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.905250 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-secret-telemeter-client\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.905435 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.905420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-telemeter-client-tls\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.905499 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.905476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d6b1ef2-cacb-40f5-917d-814193a13f11-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:33.914727 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:33.914707 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6ssh\" (UniqueName: \"kubernetes.io/projected/5d6b1ef2-cacb-40f5-917d-814193a13f11-kube-api-access-j6ssh\") pod \"telemeter-client-7877b9fcd5-4ptmf\" (UID: \"5d6b1ef2-cacb-40f5-917d-814193a13f11\") " pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:34.055450 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:34.055381 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" Jul 22 18:46:34.199202 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:34.199169 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf"] Jul 22 18:46:34.202850 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:46:34.202792 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6b1ef2_cacb_40f5_917d_814193a13f11.slice/crio-0cb069e55d40093699acd1a2c1974a4aeb883c0db4a5b0c374524a44808121da WatchSource:0}: Error finding container 0cb069e55d40093699acd1a2c1974a4aeb883c0db4a5b0c374524a44808121da: Status 404 returned error can't find the container with id 0cb069e55d40093699acd1a2c1974a4aeb883c0db4a5b0c374524a44808121da Jul 22 18:46:34.447925 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:34.447885 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" event={"ID":"5d6b1ef2-cacb-40f5-917d-814193a13f11","Type":"ContainerStarted","Data":"0cb069e55d40093699acd1a2c1974a4aeb883c0db4a5b0c374524a44808121da"} Jul 22 18:46:36.458965 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:36.458930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" event={"ID":"5d6b1ef2-cacb-40f5-917d-814193a13f11","Type":"ContainerStarted","Data":"369cd025e5bf657591d8777fea6604f354decdfe3378f8d2a1cc3f2831772996"} Jul 22 18:46:36.459275 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:36.458986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" event={"ID":"5d6b1ef2-cacb-40f5-917d-814193a13f11","Type":"ContainerStarted","Data":"753e2f60325bedb26f6dfc9cdb3d240a9b974e81a1f00203720ecde67325363e"} Jul 22 18:46:37.464981 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:37.464943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" event={"ID":"5d6b1ef2-cacb-40f5-917d-814193a13f11","Type":"ContainerStarted","Data":"fb392c653ec386baa8a78a44cf1602ff4e2c438312992563969ae53b2198c98a"} Jul 22 18:46:37.492634 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:37.492548 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7877b9fcd5-4ptmf" podStartSLOduration=2.355031726 podStartE2EDuration="4.492534507s" totalImagesPullingTime="2.137502781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:46:37.491531071 +0000 UTC m=+406.009342006" watchObservedRunningTime="2026-07-22 18:46:37.492534507 +0000 UTC m=+406.010345422" Jul 22 18:46:38.288710 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.288678 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b74b484d6-c7r7l"] Jul 22 18:46:38.291237 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.291220 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.319023 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.318993 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b74b484d6-c7r7l"] Jul 22 18:46:38.339796 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.339769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-trusted-ca-bundle\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.339933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.339808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-oauth-serving-cert\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.339933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.339863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-console-config\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.339933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.339889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b8dk\" (UniqueName: \"kubernetes.io/projected/b53adb97-b115-4518-ab88-e91b370a56b3-kube-api-access-2b8dk\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.339933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.339908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-oauth-config\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.339933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.339929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-serving-cert\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.340195 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.340051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-service-ca\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441409 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441374 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b8dk\" (UniqueName: \"kubernetes.io/projected/b53adb97-b115-4518-ab88-e91b370a56b3-kube-api-access-2b8dk\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441409 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-oauth-config\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441430 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-serving-cert\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441488 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-service-ca\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441534 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-trusted-ca-bundle\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-oauth-serving-cert\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.441626 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.441614 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-console-config\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.442388 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.442362 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-service-ca\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.442502 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.442474 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-console-config\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.442551 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.442504 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-oauth-serving-cert\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.442604 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.442580 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-trusted-ca-bundle\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.444023 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.444004 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-oauth-config\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.444145 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.444128 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-serving-cert\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.450497 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.450475 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b8dk\" (UniqueName: \"kubernetes.io/projected/b53adb97-b115-4518-ab88-e91b370a56b3-kube-api-access-2b8dk\") pod \"console-6b74b484d6-c7r7l\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.600625 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.600528 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:38.732585 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:38.732557 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b74b484d6-c7r7l"] Jul 22 18:46:38.734365 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:46:38.734335 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53adb97_b115_4518_ab88_e91b370a56b3.slice/crio-25becf7296ddfc2126fdd10eb2fc44336c4615061eb8687dde2be7a7e45b1d0f WatchSource:0}: Error finding container 25becf7296ddfc2126fdd10eb2fc44336c4615061eb8687dde2be7a7e45b1d0f: Status 404 returned error can't find the container with id 25becf7296ddfc2126fdd10eb2fc44336c4615061eb8687dde2be7a7e45b1d0f Jul 22 18:46:39.473533 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:39.473494 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b74b484d6-c7r7l" event={"ID":"b53adb97-b115-4518-ab88-e91b370a56b3","Type":"ContainerStarted","Data":"15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d"} Jul 22 18:46:39.473717 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:39.473557 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b74b484d6-c7r7l" event={"ID":"b53adb97-b115-4518-ab88-e91b370a56b3","Type":"ContainerStarted","Data":"25becf7296ddfc2126fdd10eb2fc44336c4615061eb8687dde2be7a7e45b1d0f"} Jul 22 18:46:39.497443 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:39.497396 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b74b484d6-c7r7l" podStartSLOduration=1.4973797119999999 podStartE2EDuration="1.497379712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:46:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:46:39.497154374 +0000 UTC m=+408.014965300" watchObservedRunningTime="2026-07-22 18:46:39.497379712 +0000 UTC m=+408.015190626" Jul 22 18:46:45.847981 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:45.847933 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b74b484d6-c7r7l"] Jul 22 18:46:45.885439 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:45.885407 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64474db96-68672"] Jul 22 18:46:45.888760 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:45.888738 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64474db96-68672" Jul 22 18:46:45.902103 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:45.902075 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64474db96-68672"] Jul 22 18:46:46.008212 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008169 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhwhs\" (UniqueName: \"kubernetes.io/projected/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-kube-api-access-lhwhs\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.008212 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-oauth-serving-cert\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.008534 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-oauth-config\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.008534 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-trusted-ca-bundle\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.008534 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-config\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.008534 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008505 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-serving-cert\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.008534 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.008523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-service-ca\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109050 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.108949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-serving-cert\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109050 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.108992 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-service-ca\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.109053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhwhs\" (UniqueName: \"kubernetes.io/projected/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-kube-api-access-lhwhs\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.109084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-oauth-serving-cert\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.109186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-oauth-config\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.109209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-trusted-ca-bundle\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.109282 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.109234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-config\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.110074 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.110045 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-oauth-serving-cert\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.110180 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.110127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-config\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.110223 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.110207 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-service-ca\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.110454 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.110431 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-trusted-ca-bundle\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.111746 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.111724 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-serving-cert\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.111857 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.111767 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-oauth-config\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.117327 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.117306 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhwhs\" (UniqueName: \"kubernetes.io/projected/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-kube-api-access-lhwhs\") pod \"console-64474db96-68672\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.198579 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.198544 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64474db96-68672" Jul 22 18:46:46.346461 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.346433 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64474db96-68672"] Jul 22 18:46:46.349876 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:46:46.349840 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfece5d9a_0c11_49e3_a9cd_2310e44df1c0.slice/crio-c060bc256e8f893b7522fd8ccd02412259cd6ec514bf29647bfd039e6ba94f65 WatchSource:0}: Error finding container c060bc256e8f893b7522fd8ccd02412259cd6ec514bf29647bfd039e6ba94f65: Status 404 returned error can't find the container with id c060bc256e8f893b7522fd8ccd02412259cd6ec514bf29647bfd039e6ba94f65 Jul 22 18:46:46.505696 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.505653 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64474db96-68672" event={"ID":"fece5d9a-0c11-49e3-a9cd-2310e44df1c0","Type":"ContainerStarted","Data":"0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f"} Jul 22 18:46:46.505876 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.505707 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64474db96-68672" event={"ID":"fece5d9a-0c11-49e3-a9cd-2310e44df1c0","Type":"ContainerStarted","Data":"c060bc256e8f893b7522fd8ccd02412259cd6ec514bf29647bfd039e6ba94f65"} Jul 22 18:46:46.528201 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:46.528152 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64474db96-68672" podStartSLOduration=1.528137615 podStartE2EDuration="1.528137615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:46:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:46:46.527150439 +0000 UTC m=+415.044961353" watchObservedRunningTime="2026-07-22 18:46:46.528137615 +0000 UTC m=+415.045948528" Jul 22 18:46:48.601644 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:48.601591 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:46:56.198964 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:56.198913 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64474db96-68672" Jul 22 18:46:56.199453 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:56.199002 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64474db96-68672" Jul 22 18:46:56.203912 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:56.203883 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64474db96-68672" Jul 22 18:46:56.552496 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:56.552415 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64474db96-68672" Jul 22 18:46:56.606158 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:46:56.606127 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d64b4fb88-xzndf"] Jul 22 18:47:10.868448 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:10.868377 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b74b484d6-c7r7l" podUID="b53adb97-b115-4518-ab88-e91b370a56b3" containerName="console" containerID="cri-o://15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d" gracePeriod=15 Jul 22 18:47:11.117780 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.117752 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:47:11.231086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.230996 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-serving-cert\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231041 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-service-ca\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231083 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-trusted-ca-bundle\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231105 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-oauth-config\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-oauth-serving-cert\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231143 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b8dk\" (UniqueName: \"kubernetes.io/projected/b53adb97-b115-4518-ab88-e91b370a56b3-kube-api-access-2b8dk\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231168 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-console-config\") pod \"b53adb97-b115-4518-ab88-e91b370a56b3\" (UID: \"b53adb97-b115-4518-ab88-e91b370a56b3\") " Jul 22 18:47:11.231555 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231525 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:11.231555 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231520 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:11.231764 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231736 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-console-config" (OuterVolumeSpecName: "console-config") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:11.231843 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.231798 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:11.233302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.233277 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:47:11.233372 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.233301 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b53adb97-b115-4518-ab88-e91b370a56b3-kube-api-access-2b8dk" (OuterVolumeSpecName: "kube-api-access-2b8dk") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "kube-api-access-2b8dk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:47:11.233372 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.233321 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b53adb97-b115-4518-ab88-e91b370a56b3" (UID: "b53adb97-b115-4518-ab88-e91b370a56b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:47:11.332741 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332708 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.332741 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332736 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-service-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.332741 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332745 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-trusted-ca-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.332977 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332754 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b53adb97-b115-4518-ab88-e91b370a56b3-console-oauth-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.332977 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332763 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-oauth-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.332977 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332771 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b8dk\" (UniqueName: \"kubernetes.io/projected/b53adb97-b115-4518-ab88-e91b370a56b3-kube-api-access-2b8dk\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.332977 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.332781 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b53adb97-b115-4518-ab88-e91b370a56b3-console-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:11.617023 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.616989 2566 generic.go:356] "Generic (PLEG): container finished" podID="b53adb97-b115-4518-ab88-e91b370a56b3" containerID="15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d" exitCode=0 Jul 22 18:47:11.617200 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.617033 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b74b484d6-c7r7l" Jul 22 18:47:11.617200 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.617060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b74b484d6-c7r7l" event={"ID":"b53adb97-b115-4518-ab88-e91b370a56b3","Type":"ContainerDied","Data":"15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d"} Jul 22 18:47:11.617200 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.617105 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b74b484d6-c7r7l" event={"ID":"b53adb97-b115-4518-ab88-e91b370a56b3","Type":"ContainerDied","Data":"25becf7296ddfc2126fdd10eb2fc44336c4615061eb8687dde2be7a7e45b1d0f"} Jul 22 18:47:11.617200 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.617125 2566 scope.go:117] "RemoveContainer" containerID="15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d" Jul 22 18:47:11.626303 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.626286 2566 scope.go:117] "RemoveContainer" containerID="15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d" Jul 22 18:47:11.626546 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:47:11.626527 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d\": container with ID starting with 15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d not found: ID does not exist" containerID="15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d" Jul 22 18:47:11.626586 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.626556 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d"} err="failed to get container status \"15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d\": rpc error: code = NotFound desc = could not find container \"15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d\": container with ID starting with 15bbfe325b3f4c0ed11188fd36e29cefc142d2144a4c2672a5a4e03423ea904d not found: ID does not exist" Jul 22 18:47:11.639374 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.639344 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b74b484d6-c7r7l"] Jul 22 18:47:11.643900 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:11.643870 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b74b484d6-c7r7l"] Jul 22 18:47:12.043113 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:12.043031 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b53adb97-b115-4518-ab88-e91b370a56b3" path="/var/lib/kubelet/pods/b53adb97-b115-4518-ab88-e91b370a56b3/volumes" Jul 22 18:47:21.634617 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.634508 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d64b4fb88-xzndf" podUID="0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" containerName="console" containerID="cri-o://ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a" gracePeriod=15 Jul 22 18:47:21.661103 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:47:21.661075 2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce38c2c_0e5d_4f9b_b4f9_2e03b2b523ef.slice/crio-ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a.scope\": RecentStats: unable to find data in memory cache]" Jul 22 18:47:21.884149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.884121 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:47:21.926272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926198 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wkvq\" (UniqueName: \"kubernetes.io/projected/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-kube-api-access-4wkvq\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926247 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-service-ca\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926272 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926267 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-serving-cert\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926288 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-oauth-serving-cert\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926318 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-trusted-ca-bundle\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926356 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-oauth-config\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926375 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-config\") pod \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\" (UID: \"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef\") " Jul 22 18:47:21.926805 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926743 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-service-ca" (OuterVolumeSpecName: "service-ca") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:21.926940 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926898 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:21.926940 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926911 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:21.926940 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.926929 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-config" (OuterVolumeSpecName: "console-config") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:47:21.929265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.929225 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-kube-api-access-4wkvq" (OuterVolumeSpecName: "kube-api-access-4wkvq") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "kube-api-access-4wkvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:47:21.929369 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.929236 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:47:21.929524 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:21.929499 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" (UID: "0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:47:22.027052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027010 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wkvq\" (UniqueName: \"kubernetes.io/projected/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-kube-api-access-4wkvq\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.027052 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027047 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-service-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.027265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027063 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.027265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027075 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-oauth-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.027265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027088 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-trusted-ca-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.027265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027099 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-oauth-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.027265 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.027110 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef-console-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:47:22.667332 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.667301 2566 generic.go:356] "Generic (PLEG): container finished" podID="0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" containerID="ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a" exitCode=0 Jul 22 18:47:22.667767 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.667346 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d64b4fb88-xzndf" Jul 22 18:47:22.667767 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.667394 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64b4fb88-xzndf" event={"ID":"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef","Type":"ContainerDied","Data":"ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a"} Jul 22 18:47:22.667767 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.667444 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d64b4fb88-xzndf" event={"ID":"0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef","Type":"ContainerDied","Data":"636cd88844c8afadce0734aab8add32813695fb526f2f7a4932f6b2c31f8cc5e"} Jul 22 18:47:22.667767 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.667465 2566 scope.go:117] "RemoveContainer" containerID="ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a" Jul 22 18:47:22.675861 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.675845 2566 scope.go:117] "RemoveContainer" containerID="ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a" Jul 22 18:47:22.681225 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:47:22.676579 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a\": container with ID starting with ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a not found: ID does not exist" containerID="ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a" Jul 22 18:47:22.681225 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.676617 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a"} err="failed to get container status \"ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a\": rpc error: code = NotFound desc = could not find container \"ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a\": container with ID starting with ed05b32d2988ec2bd366f86b1a93619cb505fa820be1414d239e8a0083a12a4a not found: ID does not exist" Jul 22 18:47:22.686550 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.686524 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d64b4fb88-xzndf"] Jul 22 18:47:22.695835 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:22.695800 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d64b4fb88-xzndf"] Jul 22 18:47:24.043133 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:47:24.043101 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" path="/var/lib/kubelet/pods/0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef/volumes" Jul 22 18:48:07.455992 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.455959 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7"] Jul 22 18:48:07.456419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.456273 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" containerName="console" Jul 22 18:48:07.456419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.456285 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" containerName="console" Jul 22 18:48:07.456419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.456295 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b53adb97-b115-4518-ab88-e91b370a56b3" containerName="console" Jul 22 18:48:07.456419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.456301 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b53adb97-b115-4518-ab88-e91b370a56b3" containerName="console" Jul 22 18:48:07.456419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.456367 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ce38c2c-0e5d-4f9b-b4f9-2e03b2b523ef" containerName="console" Jul 22 18:48:07.456419 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.456375 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b53adb97-b115-4518-ab88-e91b370a56b3" containerName="console" Jul 22 18:48:07.460751 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.460734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.463180 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.463156 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 18:48:07.463343 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.463230 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 18:48:07.463343 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.463261 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fxrrb\"" Jul 22 18:48:07.467997 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.467852 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7"] Jul 22 18:48:07.525229 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.525186 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.525396 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.525237 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjst6\" (UniqueName: \"kubernetes.io/projected/66300e28-7414-478e-a596-bc57b4a90732-kube-api-access-bjst6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.525396 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.525303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.626007 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.625949 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.626204 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.626020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjst6\" (UniqueName: \"kubernetes.io/projected/66300e28-7414-478e-a596-bc57b4a90732-kube-api-access-bjst6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.626204 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.626069 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.626336 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.626305 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.626431 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.626406 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.634157 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.634137 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjst6\" (UniqueName: \"kubernetes.io/projected/66300e28-7414-478e-a596-bc57b4a90732-kube-api-access-bjst6\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.770980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.770890 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:07.901875 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:07.901834 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7"] Jul 22 18:48:07.904244 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:48:07.904216 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66300e28_7414_478e_a596_bc57b4a90732.slice/crio-268ce9a7baea85a66123ccf5d490782d242557a6f47066f805cf7a46c5b7d652 WatchSource:0}: Error finding container 268ce9a7baea85a66123ccf5d490782d242557a6f47066f805cf7a46c5b7d652: Status 404 returned error can't find the container with id 268ce9a7baea85a66123ccf5d490782d242557a6f47066f805cf7a46c5b7d652 Jul 22 18:48:08.863739 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:08.863705 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" event={"ID":"66300e28-7414-478e-a596-bc57b4a90732","Type":"ContainerStarted","Data":"268ce9a7baea85a66123ccf5d490782d242557a6f47066f805cf7a46c5b7d652"} Jul 22 18:48:15.897202 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:15.897165 2566 generic.go:356] "Generic (PLEG): container finished" podID="66300e28-7414-478e-a596-bc57b4a90732" containerID="96ad99efc3fed4d9a71e5c5dc1c6aca5008c48578c95e9e29aa1f2b3f61d7787" exitCode=0 Jul 22 18:48:15.897611 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:15.897248 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" event={"ID":"66300e28-7414-478e-a596-bc57b4a90732","Type":"ContainerDied","Data":"96ad99efc3fed4d9a71e5c5dc1c6aca5008c48578c95e9e29aa1f2b3f61d7787"} Jul 22 18:48:18.911840 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:18.911790 2566 generic.go:356] "Generic (PLEG): container finished" podID="66300e28-7414-478e-a596-bc57b4a90732" containerID="66657fcf864a3fbbd1c0a11f529d6e801479c27d02188925a02c364a254bfb3e" exitCode=0 Jul 22 18:48:18.912212 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:18.911868 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" event={"ID":"66300e28-7414-478e-a596-bc57b4a90732","Type":"ContainerDied","Data":"66657fcf864a3fbbd1c0a11f529d6e801479c27d02188925a02c364a254bfb3e"} Jul 22 18:48:29.963676 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:29.963580 2566 generic.go:356] "Generic (PLEG): container finished" podID="66300e28-7414-478e-a596-bc57b4a90732" containerID="33a6ba5cf35ab8801173428f01dbc92e97608050995ebe8316995c3d0dbec83a" exitCode=0 Jul 22 18:48:29.963676 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:29.963656 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" event={"ID":"66300e28-7414-478e-a596-bc57b4a90732","Type":"ContainerDied","Data":"33a6ba5cf35ab8801173428f01dbc92e97608050995ebe8316995c3d0dbec83a"} Jul 22 18:48:31.091128 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.091101 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:31.240302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.240195 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-util\") pod \"66300e28-7414-478e-a596-bc57b4a90732\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " Jul 22 18:48:31.240302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.240276 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjst6\" (UniqueName: \"kubernetes.io/projected/66300e28-7414-478e-a596-bc57b4a90732-kube-api-access-bjst6\") pod \"66300e28-7414-478e-a596-bc57b4a90732\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " Jul 22 18:48:31.240302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.240303 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-bundle\") pod \"66300e28-7414-478e-a596-bc57b4a90732\" (UID: \"66300e28-7414-478e-a596-bc57b4a90732\") " Jul 22 18:48:31.241016 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.240986 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-bundle" (OuterVolumeSpecName: "bundle") pod "66300e28-7414-478e-a596-bc57b4a90732" (UID: "66300e28-7414-478e-a596-bc57b4a90732"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:48:31.242590 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.242559 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66300e28-7414-478e-a596-bc57b4a90732-kube-api-access-bjst6" (OuterVolumeSpecName: "kube-api-access-bjst6") pod "66300e28-7414-478e-a596-bc57b4a90732" (UID: "66300e28-7414-478e-a596-bc57b4a90732"). InnerVolumeSpecName "kube-api-access-bjst6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:48:31.244250 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.244230 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-util" (OuterVolumeSpecName: "util") pod "66300e28-7414-478e-a596-bc57b4a90732" (UID: "66300e28-7414-478e-a596-bc57b4a90732"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:48:31.341507 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.341464 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjst6\" (UniqueName: \"kubernetes.io/projected/66300e28-7414-478e-a596-bc57b4a90732-kube-api-access-bjst6\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:48:31.341507 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.341504 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:48:31.341507 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.341513 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/66300e28-7414-478e-a596-bc57b4a90732-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:48:31.974117 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.974077 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" event={"ID":"66300e28-7414-478e-a596-bc57b4a90732","Type":"ContainerDied","Data":"268ce9a7baea85a66123ccf5d490782d242557a6f47066f805cf7a46c5b7d652"} Jul 22 18:48:31.974117 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.974114 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nhql7" Jul 22 18:48:31.974309 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:31.974132 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="268ce9a7baea85a66123ccf5d490782d242557a6f47066f805cf7a46c5b7d652" Jul 22 18:48:35.434607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434573 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58"] Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434914 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="extract" Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434926 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="extract" Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434934 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="pull" Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434939 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="pull" Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434950 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="util" Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.434955 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="util" Jul 22 18:48:35.435037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.435009 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="66300e28-7414-478e-a596-bc57b4a90732" containerName="extract" Jul 22 18:48:35.437056 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.437040 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.439515 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.439490 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:48:35.439667 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.439490 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gn82c\"" Jul 22 18:48:35.439867 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.439854 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 18:48:35.452958 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.452931 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58"] Jul 22 18:48:35.577068 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.577029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e96a765f-fe27-4a56-ac7f-94ba43161e21-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-plj58\" (UID: \"e96a765f-fe27-4a56-ac7f-94ba43161e21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.577245 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.577133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcptx\" (UniqueName: \"kubernetes.io/projected/e96a765f-fe27-4a56-ac7f-94ba43161e21-kube-api-access-xcptx\") pod \"cert-manager-operator-controller-manager-55c444bf86-plj58\" (UID: \"e96a765f-fe27-4a56-ac7f-94ba43161e21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.677741 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.677699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e96a765f-fe27-4a56-ac7f-94ba43161e21-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-plj58\" (UID: \"e96a765f-fe27-4a56-ac7f-94ba43161e21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.677940 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.677790 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcptx\" (UniqueName: \"kubernetes.io/projected/e96a765f-fe27-4a56-ac7f-94ba43161e21-kube-api-access-xcptx\") pod \"cert-manager-operator-controller-manager-55c444bf86-plj58\" (UID: \"e96a765f-fe27-4a56-ac7f-94ba43161e21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.678104 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.678082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e96a765f-fe27-4a56-ac7f-94ba43161e21-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-plj58\" (UID: \"e96a765f-fe27-4a56-ac7f-94ba43161e21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.689139 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.689077 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcptx\" (UniqueName: \"kubernetes.io/projected/e96a765f-fe27-4a56-ac7f-94ba43161e21-kube-api-access-xcptx\") pod \"cert-manager-operator-controller-manager-55c444bf86-plj58\" (UID: \"e96a765f-fe27-4a56-ac7f-94ba43161e21\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.746103 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.746072 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" Jul 22 18:48:35.876567 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.876544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58"] Jul 22 18:48:35.879082 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:48:35.879054 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96a765f_fe27_4a56_ac7f_94ba43161e21.slice/crio-d929ae0343488c506d1c472f4c6e6119d13aa6f352f56a23c72e32b2f7756721 WatchSource:0}: Error finding container d929ae0343488c506d1c472f4c6e6119d13aa6f352f56a23c72e32b2f7756721: Status 404 returned error can't find the container with id d929ae0343488c506d1c472f4c6e6119d13aa6f352f56a23c72e32b2f7756721 Jul 22 18:48:35.990668 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:35.990573 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" event={"ID":"e96a765f-fe27-4a56-ac7f-94ba43161e21","Type":"ContainerStarted","Data":"d929ae0343488c506d1c472f4c6e6119d13aa6f352f56a23c72e32b2f7756721"} Jul 22 18:48:38.001555 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.001521 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" event={"ID":"e96a765f-fe27-4a56-ac7f-94ba43161e21","Type":"ContainerStarted","Data":"aff958f64fe2dac113e46b9a38ce867761a625e7757f4694b8b1887d22eb66d0"} Jul 22 18:48:38.027809 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.027753 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-plj58" podStartSLOduration=1.586464104 podStartE2EDuration="3.027737548s" totalImagesPullingTime="1.441273444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:48:38.025960751 +0000 UTC m=+526.543771666" watchObservedRunningTime="2026-07-22 18:48:38.027737548 +0000 UTC m=+526.545548461" Jul 22 18:48:38.130759 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.130724 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wrvxf"] Jul 22 18:48:38.132853 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.132835 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.135174 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.135142 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 18:48:38.135297 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.135194 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 18:48:38.143377 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.143357 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wrvxf"] Jul 22 18:48:38.203011 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.202985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddhkk\" (UniqueName: \"kubernetes.io/projected/4b77c216-588e-46e5-a980-79e0600e0545-kube-api-access-ddhkk\") pod \"cert-manager-webhook-6486ffd997-wrvxf\" (UID: \"4b77c216-588e-46e5-a980-79e0600e0545\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.203188 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.203016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b77c216-588e-46e5-a980-79e0600e0545-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wrvxf\" (UID: \"4b77c216-588e-46e5-a980-79e0600e0545\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.303994 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.303900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddhkk\" (UniqueName: \"kubernetes.io/projected/4b77c216-588e-46e5-a980-79e0600e0545-kube-api-access-ddhkk\") pod \"cert-manager-webhook-6486ffd997-wrvxf\" (UID: \"4b77c216-588e-46e5-a980-79e0600e0545\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.303994 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.303950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b77c216-588e-46e5-a980-79e0600e0545-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wrvxf\" (UID: \"4b77c216-588e-46e5-a980-79e0600e0545\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.314964 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.314937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4b77c216-588e-46e5-a980-79e0600e0545-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wrvxf\" (UID: \"4b77c216-588e-46e5-a980-79e0600e0545\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.315062 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.315021 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddhkk\" (UniqueName: \"kubernetes.io/projected/4b77c216-588e-46e5-a980-79e0600e0545-kube-api-access-ddhkk\") pod \"cert-manager-webhook-6486ffd997-wrvxf\" (UID: \"4b77c216-588e-46e5-a980-79e0600e0545\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.455414 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.455375 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:38.612977 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:38.612953 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wrvxf"] Jul 22 18:48:38.614616 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:48:38.614591 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b77c216_588e_46e5_a980_79e0600e0545.slice/crio-502f7803121d59b426d74a50a8f5049c9f5cd0859e4d9cd4689d25e6ceb5f5b8 WatchSource:0}: Error finding container 502f7803121d59b426d74a50a8f5049c9f5cd0859e4d9cd4689d25e6ceb5f5b8: Status 404 returned error can't find the container with id 502f7803121d59b426d74a50a8f5049c9f5cd0859e4d9cd4689d25e6ceb5f5b8 Jul 22 18:48:39.012520 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.012481 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" event={"ID":"4b77c216-588e-46e5-a980-79e0600e0545","Type":"ContainerStarted","Data":"502f7803121d59b426d74a50a8f5049c9f5cd0859e4d9cd4689d25e6ceb5f5b8"} Jul 22 18:48:39.714093 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.714061 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq"] Jul 22 18:48:39.717355 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.717337 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.719812 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.719790 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 18:48:39.719919 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.719799 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 18:48:39.720372 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.720349 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fxrrb\"" Jul 22 18:48:39.727560 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.727538 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq"] Jul 22 18:48:39.816239 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.816202 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb2wt\" (UniqueName: \"kubernetes.io/projected/e5e7755a-beb0-4e51-95b6-bec790238e11-kube-api-access-jb2wt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.816418 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.816262 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.816418 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.816342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.917758 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.917718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.917933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.917795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.917933 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.917854 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb2wt\" (UniqueName: \"kubernetes.io/projected/e5e7755a-beb0-4e51-95b6-bec790238e11-kube-api-access-jb2wt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.918150 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.918126 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.918217 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.918164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:39.926004 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:39.925977 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb2wt\" (UniqueName: \"kubernetes.io/projected/e5e7755a-beb0-4e51-95b6-bec790238e11-kube-api-access-jb2wt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:40.027980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:40.027899 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:40.153631 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:40.153597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq"] Jul 22 18:48:40.156378 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:48:40.156349 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e7755a_beb0_4e51_95b6_bec790238e11.slice/crio-882cef71d6a1feeed80715d0fa9955e4e0770bc2192347609c04bef94ccbe411 WatchSource:0}: Error finding container 882cef71d6a1feeed80715d0fa9955e4e0770bc2192347609c04bef94ccbe411: Status 404 returned error can't find the container with id 882cef71d6a1feeed80715d0fa9955e4e0770bc2192347609c04bef94ccbe411 Jul 22 18:48:41.022367 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:41.022329 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerID="ece010689a21fded2dd7d6adc3cbfc1e31e7732e0ef9cc782d7806da4a3e5f7c" exitCode=0 Jul 22 18:48:41.022521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:41.022421 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" event={"ID":"e5e7755a-beb0-4e51-95b6-bec790238e11","Type":"ContainerDied","Data":"ece010689a21fded2dd7d6adc3cbfc1e31e7732e0ef9cc782d7806da4a3e5f7c"} Jul 22 18:48:41.022521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:41.022485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" event={"ID":"e5e7755a-beb0-4e51-95b6-bec790238e11","Type":"ContainerStarted","Data":"882cef71d6a1feeed80715d0fa9955e4e0770bc2192347609c04bef94ccbe411"} Jul 22 18:48:44.040045 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:44.040014 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerID="88897d588348d612602b714f7e840198254cbe85173245dd86411c62206f94fa" exitCode=0 Jul 22 18:48:44.044121 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:44.044095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" event={"ID":"e5e7755a-beb0-4e51-95b6-bec790238e11","Type":"ContainerDied","Data":"88897d588348d612602b714f7e840198254cbe85173245dd86411c62206f94fa"} Jul 22 18:48:45.047001 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:45.046919 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerID="d26036bd266492bec43469fdf40140452008828bc02cffe0b4ab98bea5028cfb" exitCode=0 Jul 22 18:48:45.047356 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:45.047014 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" event={"ID":"e5e7755a-beb0-4e51-95b6-bec790238e11","Type":"ContainerDied","Data":"d26036bd266492bec43469fdf40140452008828bc02cffe0b4ab98bea5028cfb"} Jul 22 18:48:46.200221 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.200190 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:46.385538 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.385509 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb2wt\" (UniqueName: \"kubernetes.io/projected/e5e7755a-beb0-4e51-95b6-bec790238e11-kube-api-access-jb2wt\") pod \"e5e7755a-beb0-4e51-95b6-bec790238e11\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " Jul 22 18:48:46.385716 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.385581 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-util\") pod \"e5e7755a-beb0-4e51-95b6-bec790238e11\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " Jul 22 18:48:46.385716 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.385650 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-bundle\") pod \"e5e7755a-beb0-4e51-95b6-bec790238e11\" (UID: \"e5e7755a-beb0-4e51-95b6-bec790238e11\") " Jul 22 18:48:46.386086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.386063 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-bundle" (OuterVolumeSpecName: "bundle") pod "e5e7755a-beb0-4e51-95b6-bec790238e11" (UID: "e5e7755a-beb0-4e51-95b6-bec790238e11"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:48:46.387883 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.387841 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5e7755a-beb0-4e51-95b6-bec790238e11-kube-api-access-jb2wt" (OuterVolumeSpecName: "kube-api-access-jb2wt") pod "e5e7755a-beb0-4e51-95b6-bec790238e11" (UID: "e5e7755a-beb0-4e51-95b6-bec790238e11"). InnerVolumeSpecName "kube-api-access-jb2wt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:48:46.392932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.392890 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-util" (OuterVolumeSpecName: "util") pod "e5e7755a-beb0-4e51-95b6-bec790238e11" (UID: "e5e7755a-beb0-4e51-95b6-bec790238e11"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:48:46.487197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.487160 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb2wt\" (UniqueName: \"kubernetes.io/projected/e5e7755a-beb0-4e51-95b6-bec790238e11-kube-api-access-jb2wt\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:48:46.487197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.487192 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:48:46.487197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:46.487203 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e5e7755a-beb0-4e51-95b6-bec790238e11-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:48:47.059405 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:47.059370 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" Jul 22 18:48:47.059579 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:47.059372 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmgbwq" event={"ID":"e5e7755a-beb0-4e51-95b6-bec790238e11","Type":"ContainerDied","Data":"882cef71d6a1feeed80715d0fa9955e4e0770bc2192347609c04bef94ccbe411"} Jul 22 18:48:47.059579 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:47.059485 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="882cef71d6a1feeed80715d0fa9955e4e0770bc2192347609c04bef94ccbe411" Jul 22 18:48:48.064924 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:48.064840 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" event={"ID":"4b77c216-588e-46e5-a980-79e0600e0545","Type":"ContainerStarted","Data":"3c44c5959e4fea630eb694011a6dbc700372190afdba2473f8e2e6b241fc3aa5"} Jul 22 18:48:48.065264 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:48.064986 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:48:48.081245 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:48.081199 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" podStartSLOduration=0.968901576 podStartE2EDuration="10.081184945s" totalImagesPullingTime="9.112283369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:48:48.080431276 +0000 UTC m=+536.598242204" watchObservedRunningTime="2026-07-22 18:48:48.081184945 +0000 UTC m=+536.598995859" Jul 22 18:48:49.942796 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.942764 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k64nm"] Jul 22 18:48:49.943184 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943095 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="util" Jul 22 18:48:49.943184 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943107 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="util" Jul 22 18:48:49.943184 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943115 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="extract" Jul 22 18:48:49.943184 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943122 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="extract" Jul 22 18:48:49.943184 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943139 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="pull" Jul 22 18:48:49.943184 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943145 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="pull" Jul 22 18:48:49.943364 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.943192 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5e7755a-beb0-4e51-95b6-bec790238e11" containerName="extract" Jul 22 18:48:49.946164 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.946147 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:49.948392 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.948371 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pjt8b\"" Jul 22 18:48:49.955954 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:49.955932 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k64nm"] Jul 22 18:48:50.117755 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.117720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gjs5\" (UniqueName: \"kubernetes.io/projected/d267a88b-be29-48ce-8b33-1a5cb58a7352-kube-api-access-8gjs5\") pod \"cert-manager-55b4d954d5-k64nm\" (UID: \"d267a88b-be29-48ce-8b33-1a5cb58a7352\") " pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.117755 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.117759 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d267a88b-be29-48ce-8b33-1a5cb58a7352-bound-sa-token\") pod \"cert-manager-55b4d954d5-k64nm\" (UID: \"d267a88b-be29-48ce-8b33-1a5cb58a7352\") " pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.218936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.218863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gjs5\" (UniqueName: \"kubernetes.io/projected/d267a88b-be29-48ce-8b33-1a5cb58a7352-kube-api-access-8gjs5\") pod \"cert-manager-55b4d954d5-k64nm\" (UID: \"d267a88b-be29-48ce-8b33-1a5cb58a7352\") " pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.218936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.218899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d267a88b-be29-48ce-8b33-1a5cb58a7352-bound-sa-token\") pod \"cert-manager-55b4d954d5-k64nm\" (UID: \"d267a88b-be29-48ce-8b33-1a5cb58a7352\") " pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.226938 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.226909 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d267a88b-be29-48ce-8b33-1a5cb58a7352-bound-sa-token\") pod \"cert-manager-55b4d954d5-k64nm\" (UID: \"d267a88b-be29-48ce-8b33-1a5cb58a7352\") " pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.227042 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.226982 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gjs5\" (UniqueName: \"kubernetes.io/projected/d267a88b-be29-48ce-8b33-1a5cb58a7352-kube-api-access-8gjs5\") pod \"cert-manager-55b4d954d5-k64nm\" (UID: \"d267a88b-be29-48ce-8b33-1a5cb58a7352\") " pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.254944 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.254910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k64nm" Jul 22 18:48:50.384601 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:50.384577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k64nm"] Jul 22 18:48:50.386581 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:48:50.386553 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd267a88b_be29_48ce_8b33_1a5cb58a7352.slice/crio-f223b0ac0db0949d2ed5d129b5d95fb2e602df3b00295e359fd775da2303f649 WatchSource:0}: Error finding container f223b0ac0db0949d2ed5d129b5d95fb2e602df3b00295e359fd775da2303f649: Status 404 returned error can't find the container with id f223b0ac0db0949d2ed5d129b5d95fb2e602df3b00295e359fd775da2303f649 Jul 22 18:48:51.079720 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:51.079681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k64nm" event={"ID":"d267a88b-be29-48ce-8b33-1a5cb58a7352","Type":"ContainerStarted","Data":"cc3ae3a9e6e14fc3942c5b30f5ea53537c878e1a3840b3441c72f930e91f9fd5"} Jul 22 18:48:51.079720 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:51.079726 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k64nm" event={"ID":"d267a88b-be29-48ce-8b33-1a5cb58a7352","Type":"ContainerStarted","Data":"f223b0ac0db0949d2ed5d129b5d95fb2e602df3b00295e359fd775da2303f649"} Jul 22 18:48:51.095498 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:51.095449 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-k64nm" podStartSLOduration=2.095435894 podStartE2EDuration="2.095435894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:48:51.093382718 +0000 UTC m=+539.611193630" watchObservedRunningTime="2026-07-22 18:48:51.095435894 +0000 UTC m=+539.613246807" Jul 22 18:48:54.071379 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:48:54.071346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wrvxf" Jul 22 18:49:00.789669 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.789628 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t"] Jul 22 18:49:00.826929 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.826893 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t"] Jul 22 18:49:00.827086 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.827010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:00.829565 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.829536 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 18:49:00.830358 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.830339 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 18:49:00.830358 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.830357 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fxrrb\"" Jul 22 18:49:00.911604 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.911564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:00.911795 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.911635 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:00.911897 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:00.911800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xdm\" (UniqueName: \"kubernetes.io/projected/95c4c48f-969d-4f5c-8674-bb44419ce801-kube-api-access-24xdm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.012847 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.012779 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xdm\" (UniqueName: \"kubernetes.io/projected/95c4c48f-969d-4f5c-8674-bb44419ce801-kube-api-access-24xdm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.013046 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.012879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.013046 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.012933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.013276 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.013253 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.013343 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.013293 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.020908 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.020882 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xdm\" (UniqueName: \"kubernetes.io/projected/95c4c48f-969d-4f5c-8674-bb44419ce801-kube-api-access-24xdm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.136242 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.136210 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:01.292416 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:01.292386 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t"] Jul 22 18:49:01.295119 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:49:01.295089 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c4c48f_969d_4f5c_8674_bb44419ce801.slice/crio-cce9de52e3c86d8f9dac931ef8ec252c8cde2a0701152b1173dd4419e7b7b2b6 WatchSource:0}: Error finding container cce9de52e3c86d8f9dac931ef8ec252c8cde2a0701152b1173dd4419e7b7b2b6: Status 404 returned error can't find the container with id cce9de52e3c86d8f9dac931ef8ec252c8cde2a0701152b1173dd4419e7b7b2b6 Jul 22 18:49:02.133809 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:02.133770 2566 generic.go:356] "Generic (PLEG): container finished" podID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerID="88ced0278721fd43882aa41504f550261c6c7f7e085df8ea6e9d56858203bad0" exitCode=0 Jul 22 18:49:02.134291 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:02.133834 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" event={"ID":"95c4c48f-969d-4f5c-8674-bb44419ce801","Type":"ContainerDied","Data":"88ced0278721fd43882aa41504f550261c6c7f7e085df8ea6e9d56858203bad0"} Jul 22 18:49:02.134291 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:02.133884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" event={"ID":"95c4c48f-969d-4f5c-8674-bb44419ce801","Type":"ContainerStarted","Data":"cce9de52e3c86d8f9dac931ef8ec252c8cde2a0701152b1173dd4419e7b7b2b6"} Jul 22 18:49:03.140566 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:03.140534 2566 generic.go:356] "Generic (PLEG): container finished" podID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerID="8d02427426bcfd0494c42b58c9d3bb693eda26d5d1ad31697a1998f0594fd257" exitCode=0 Jul 22 18:49:03.141103 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:03.140616 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" event={"ID":"95c4c48f-969d-4f5c-8674-bb44419ce801","Type":"ContainerDied","Data":"8d02427426bcfd0494c42b58c9d3bb693eda26d5d1ad31697a1998f0594fd257"} Jul 22 18:49:04.147264 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:04.147230 2566 generic.go:356] "Generic (PLEG): container finished" podID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerID="e6259a21a240b4a238a25e53b9a745e070900b90d3003c2c55dad2280b24ec4c" exitCode=0 Jul 22 18:49:04.147650 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:04.147316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" event={"ID":"95c4c48f-969d-4f5c-8674-bb44419ce801","Type":"ContainerDied","Data":"e6259a21a240b4a238a25e53b9a745e070900b90d3003c2c55dad2280b24ec4c"} Jul 22 18:49:05.293936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.293911 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:05.352783 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.352743 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24xdm\" (UniqueName: \"kubernetes.io/projected/95c4c48f-969d-4f5c-8674-bb44419ce801-kube-api-access-24xdm\") pod \"95c4c48f-969d-4f5c-8674-bb44419ce801\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " Jul 22 18:49:05.352988 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.352843 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-util\") pod \"95c4c48f-969d-4f5c-8674-bb44419ce801\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " Jul 22 18:49:05.352988 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.352871 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-bundle\") pod \"95c4c48f-969d-4f5c-8674-bb44419ce801\" (UID: \"95c4c48f-969d-4f5c-8674-bb44419ce801\") " Jul 22 18:49:05.353577 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.353551 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-bundle" (OuterVolumeSpecName: "bundle") pod "95c4c48f-969d-4f5c-8674-bb44419ce801" (UID: "95c4c48f-969d-4f5c-8674-bb44419ce801"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:49:05.355039 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.355016 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c4c48f-969d-4f5c-8674-bb44419ce801-kube-api-access-24xdm" (OuterVolumeSpecName: "kube-api-access-24xdm") pod "95c4c48f-969d-4f5c-8674-bb44419ce801" (UID: "95c4c48f-969d-4f5c-8674-bb44419ce801"). InnerVolumeSpecName "kube-api-access-24xdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:49:05.358334 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.358301 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-util" (OuterVolumeSpecName: "util") pod "95c4c48f-969d-4f5c-8674-bb44419ce801" (UID: "95c4c48f-969d-4f5c-8674-bb44419ce801"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:49:05.453810 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.453705 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24xdm\" (UniqueName: \"kubernetes.io/projected/95c4c48f-969d-4f5c-8674-bb44419ce801-kube-api-access-24xdm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:49:05.453810 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.453753 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:49:05.453810 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:05.453764 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95c4c48f-969d-4f5c-8674-bb44419ce801-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:49:06.159284 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:06.159249 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" Jul 22 18:49:06.159442 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:06.159258 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aftqr4t" event={"ID":"95c4c48f-969d-4f5c-8674-bb44419ce801","Type":"ContainerDied","Data":"cce9de52e3c86d8f9dac931ef8ec252c8cde2a0701152b1173dd4419e7b7b2b6"} Jul 22 18:49:06.159442 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:06.159378 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cce9de52e3c86d8f9dac931ef8ec252c8cde2a0701152b1173dd4419e7b7b2b6" Jul 22 18:49:17.201513 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201477 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7"] Jul 22 18:49:17.201935 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201809 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="pull" Jul 22 18:49:17.201935 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201840 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="pull" Jul 22 18:49:17.201935 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201857 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="extract" Jul 22 18:49:17.201935 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201864 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="extract" Jul 22 18:49:17.201935 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201874 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="util" Jul 22 18:49:17.201935 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201880 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="util" Jul 22 18:49:17.202129 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.201942 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c4c48f-969d-4f5c-8674-bb44419ce801" containerName="extract" Jul 22 18:49:17.203738 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.203720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.206455 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.206429 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 18:49:17.206583 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.206476 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 18:49:17.207463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.207437 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fx7kj\"" Jul 22 18:49:17.207543 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.207489 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 18:49:17.207663 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.207640 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 18:49:17.207756 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.207733 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 18:49:17.222300 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.222271 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7"] Jul 22 18:49:17.357483 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.357445 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-metrics-cert\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.357657 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.357499 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wwx6\" (UniqueName: \"kubernetes.io/projected/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-kube-api-access-8wwx6\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.357657 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.357560 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-manager-config\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.357657 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.357588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-cert\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.458616 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.458529 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-metrics-cert\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.458616 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.458577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wwx6\" (UniqueName: \"kubernetes.io/projected/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-kube-api-access-8wwx6\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.458616 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.458607 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-manager-config\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.458849 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.458624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-cert\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.459380 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.459357 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-manager-config\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.461191 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.461157 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-cert\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.461289 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.461162 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-metrics-cert\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.473313 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.473288 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wwx6\" (UniqueName: \"kubernetes.io/projected/7ccc85ee-e426-4f50-a57e-e36a5c9bd833-kube-api-access-8wwx6\") pod \"lws-controller-manager-5b444cc6d8-jw2c7\" (UID: \"7ccc85ee-e426-4f50-a57e-e36a5c9bd833\") " pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.514493 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.514455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:17.651837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.651790 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7"] Jul 22 18:49:17.653979 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:49:17.653952 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ccc85ee_e426_4f50_a57e_e36a5c9bd833.slice/crio-29678867261f1fcad0bfa15e28e61ba99d3b96bfc6017bfaacae519fa50372bd WatchSource:0}: Error finding container 29678867261f1fcad0bfa15e28e61ba99d3b96bfc6017bfaacae519fa50372bd: Status 404 returned error can't find the container with id 29678867261f1fcad0bfa15e28e61ba99d3b96bfc6017bfaacae519fa50372bd Jul 22 18:49:17.997350 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:17.997313 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k"] Jul 22 18:49:18.000364 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.000341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.005915 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.005894 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bkd9t\"" Jul 22 18:49:18.006322 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.006305 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 18:49:18.006431 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.006414 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 18:49:18.006723 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.006694 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 18:49:18.006812 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.006769 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 18:49:18.048578 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.048547 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k"] Jul 22 18:49:18.064318 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.064288 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-tmp\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.064427 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.064384 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.064427 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.064420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-manifests\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.064494 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.064453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tbqn\" (UniqueName: \"kubernetes.io/projected/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-kube-api-access-5tbqn\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.064494 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.064473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.165509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.165481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.165655 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.165516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-manifests\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.165655 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.165541 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tbqn\" (UniqueName: \"kubernetes.io/projected/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-kube-api-access-5tbqn\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.165655 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.165560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.165655 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.165612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-tmp\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.165979 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.165960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-manifests\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.167839 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.167800 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-tmp\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.167953 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.167933 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.168041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.168023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-webhook-cert\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.177554 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.177526 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tbqn\" (UniqueName: \"kubernetes.io/projected/e1eb87b9-1554-49e5-a127-14de8f3ff8bf-kube-api-access-5tbqn\") pod \"opendatahub-operator-controller-manager-f9bb58876-ftm8k\" (UID: \"e1eb87b9-1554-49e5-a127-14de8f3ff8bf\") " pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.217064 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.217041 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" event={"ID":"7ccc85ee-e426-4f50-a57e-e36a5c9bd833","Type":"ContainerStarted","Data":"29678867261f1fcad0bfa15e28e61ba99d3b96bfc6017bfaacae519fa50372bd"} Jul 22 18:49:18.310919 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.310858 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:18.445864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:18.445763 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k"] Jul 22 18:49:18.449388 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:49:18.449356 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1eb87b9_1554_49e5_a127_14de8f3ff8bf.slice/crio-476c73b6f663591b883400eefbcebb59c4417c77212389f48a5c95b598a4d2d5 WatchSource:0}: Error finding container 476c73b6f663591b883400eefbcebb59c4417c77212389f48a5c95b598a4d2d5: Status 404 returned error can't find the container with id 476c73b6f663591b883400eefbcebb59c4417c77212389f48a5c95b598a4d2d5 Jul 22 18:49:19.224774 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:19.224722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" event={"ID":"e1eb87b9-1554-49e5-a127-14de8f3ff8bf","Type":"ContainerStarted","Data":"476c73b6f663591b883400eefbcebb59c4417c77212389f48a5c95b598a4d2d5"} Jul 22 18:49:20.233316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:20.233237 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" event={"ID":"7ccc85ee-e426-4f50-a57e-e36a5c9bd833","Type":"ContainerStarted","Data":"4e836bebe1375712fc87ac378afd9cf253ec3ec2dd9e349dcf7af74e59770b26"} Jul 22 18:49:20.233316 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:20.233315 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:20.249864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:20.249767 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" podStartSLOduration=1.5951236020000001 podStartE2EDuration="3.249748084s" totalImagesPullingTime="1.654624482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:20.249398442 +0000 UTC m=+568.767209356" watchObservedRunningTime="2026-07-22 18:49:20.249748084 +0000 UTC m=+568.767559001" Jul 22 18:49:22.244804 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:22.244759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" event={"ID":"e1eb87b9-1554-49e5-a127-14de8f3ff8bf","Type":"ContainerStarted","Data":"f8041a655d6b7ea65c2e0f027ac3c280b4293b9306b263ac78157e5e1056be21"} Jul 22 18:49:23.252476 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:23.252440 2566 generic.go:356] "Generic (PLEG): container finished" podID="e1eb87b9-1554-49e5-a127-14de8f3ff8bf" containerID="f8041a655d6b7ea65c2e0f027ac3c280b4293b9306b263ac78157e5e1056be21" exitCode=0 Jul 22 18:49:23.252966 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:23.252498 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" event={"ID":"e1eb87b9-1554-49e5-a127-14de8f3ff8bf","Type":"ContainerDied","Data":"f8041a655d6b7ea65c2e0f027ac3c280b4293b9306b263ac78157e5e1056be21"} Jul 22 18:49:24.259298 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:24.259262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" event={"ID":"e1eb87b9-1554-49e5-a127-14de8f3ff8bf","Type":"ContainerStarted","Data":"c8356c4afe58d77e0020110e751917ebf26f104fe93379c105816ad290207b0a"} Jul 22 18:49:24.259686 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:24.259418 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:24.283751 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:24.283698 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" podStartSLOduration=2.390427743 podStartE2EDuration="7.283682483s" totalImagesPullingTime="3.875805953s" totalInitContainerRuntime="1.017448787s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:24.28207598 +0000 UTC m=+572.799886893" watchObservedRunningTime="2026-07-22 18:49:24.283682483 +0000 UTC m=+572.801493396" Jul 22 18:49:31.240591 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:31.240554 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b444cc6d8-jw2c7" Jul 22 18:49:35.267171 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:35.267141 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f9bb58876-ftm8k" Jul 22 18:49:51.073143 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.073101 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4"] Jul 22 18:49:51.075672 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.075654 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.078312 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.078284 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ffl8l\"" Jul 22 18:49:51.078409 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.078317 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 18:49:51.088085 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.088058 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4"] Jul 22 18:49:51.159636 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159606 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhfjq\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-kube-api-access-zhfjq\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159748 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159748 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159843 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159843 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159770 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159843 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159787 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159945 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159945 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.159945 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.159889 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7abc826d-74de-44ee-8faa-a203322c7c9b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260713 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260679 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260713 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260974 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260974 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260760 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260974 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260879 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260974 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260932 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.260974 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.260961 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7abc826d-74de-44ee-8faa-a203322c7c9b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261224 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhfjq\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-kube-api-access-zhfjq\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261224 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261224 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261133 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261488 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261469 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261562 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261503 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261601 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261553 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.261880 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.261856 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7abc826d-74de-44ee-8faa-a203322c7c9b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.263266 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.263241 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.263638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.263622 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.269649 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.269628 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.270070 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.270048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhfjq\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-kube-api-access-zhfjq\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-swxw4\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.386141 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.386111 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:51.527410 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.527377 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4"] Jul 22 18:49:51.529564 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:49:51.529533 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7abc826d_74de_44ee_8faa_a203322c7c9b.slice/crio-018ba28ac01b3aa4b10beead7e9598b3f7903705917dd3213fa121dd7f60413f WatchSource:0}: Error finding container 018ba28ac01b3aa4b10beead7e9598b3f7903705917dd3213fa121dd7f60413f: Status 404 returned error can't find the container with id 018ba28ac01b3aa4b10beead7e9598b3f7903705917dd3213fa121dd7f60413f Jul 22 18:49:51.959266 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.959236 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:49:51.961216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.961193 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:49:51.965482 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.965460 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:49:51.967302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:51.967279 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:49:52.140868 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.140841 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl"] Jul 22 18:49:52.144140 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.144122 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.146337 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.146314 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 18:49:52.146440 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.146356 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 18:49:52.146440 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.146369 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 18:49:52.146550 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.146474 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cx66g\"" Jul 22 18:49:52.156314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.156293 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl"] Jul 22 18:49:52.269670 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.269594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f630aa9-42ed-40ad-befb-a325058d93be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.269670 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.269630 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.269899 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.269718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f630aa9-42ed-40ad-befb-a325058d93be-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.269899 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.269793 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.269899 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.269859 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf2sq\" (UniqueName: \"kubernetes.io/projected/7f630aa9-42ed-40ad-befb-a325058d93be-kube-api-access-lf2sq\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.269899 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.269890 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f630aa9-42ed-40ad-befb-a325058d93be-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.270089 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.270013 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371353 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f630aa9-42ed-40ad-befb-a325058d93be-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf2sq\" (UniqueName: \"kubernetes.io/projected/7f630aa9-42ed-40ad-befb-a325058d93be-kube-api-access-lf2sq\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371430 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f630aa9-42ed-40ad-befb-a325058d93be-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f630aa9-42ed-40ad-befb-a325058d93be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.371703 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.371538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.372461 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.372407 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7f630aa9-42ed-40ad-befb-a325058d93be-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.374263 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.374218 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f630aa9-42ed-40ad-befb-a325058d93be-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.374500 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.374454 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.374500 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.374459 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.374939 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.374917 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7f630aa9-42ed-40ad-befb-a325058d93be-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.380110 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.380090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f630aa9-42ed-40ad-befb-a325058d93be-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.380296 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.380277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf2sq\" (UniqueName: \"kubernetes.io/projected/7f630aa9-42ed-40ad-befb-a325058d93be-kube-api-access-lf2sq\") pod \"istiod-openshift-gateway-94bb8fbfd-p8pwl\" (UID: \"7f630aa9-42ed-40ad-befb-a325058d93be\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.397640 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.397609 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" event={"ID":"7abc826d-74de-44ee-8faa-a203322c7c9b","Type":"ContainerStarted","Data":"018ba28ac01b3aa4b10beead7e9598b3f7903705917dd3213fa121dd7f60413f"} Jul 22 18:49:52.453989 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.453950 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:49:52.618603 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:52.618580 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl"] Jul 22 18:49:52.621118 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:49:52.621083 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f630aa9_42ed_40ad_befb_a325058d93be.slice/crio-dffbcb1455b25e85961a48bacf43d084144cc470edd7f93268a2a26fbf7cdde6 WatchSource:0}: Error finding container dffbcb1455b25e85961a48bacf43d084144cc470edd7f93268a2a26fbf7cdde6: Status 404 returned error can't find the container with id dffbcb1455b25e85961a48bacf43d084144cc470edd7f93268a2a26fbf7cdde6 Jul 22 18:49:53.403748 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:53.403708 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" event={"ID":"7f630aa9-42ed-40ad-befb-a325058d93be","Type":"ContainerStarted","Data":"dffbcb1455b25e85961a48bacf43d084144cc470edd7f93268a2a26fbf7cdde6"} Jul 22 18:49:55.104733 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:55.104691 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 18:49:55.105037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:55.104772 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 18:49:55.105037 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:55.104811 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 18:49:55.415056 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:55.415012 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" event={"ID":"7abc826d-74de-44ee-8faa-a203322c7c9b","Type":"ContainerStarted","Data":"7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6"} Jul 22 18:49:55.438650 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:55.438467 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" podStartSLOduration=0.865974686 podStartE2EDuration="4.438444846s" totalImagesPullingTime="3.57247016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:49:55.436400803 +0000 UTC m=+603.954211719" watchObservedRunningTime="2026-07-22 18:49:55.438444846 +0000 UTC m=+603.956255763" Jul 22 18:49:56.387249 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:56.387213 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:49:56.388675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:56.388651 2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 18:49:56.388727 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:56.388703 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 18:49:56.802402 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:56.802315 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4"] Jul 22 18:49:57.387099 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:57.387062 2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 18:49:57.387285 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:57.387129 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 18:49:58.386675 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:58.386636 2566 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 22 18:49:58.386880 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:58.386699 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 22 18:49:58.429968 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:49:58.429792 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" containerID="cri-o://7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6" gracePeriod=30 Jul 22 18:50:00.294337 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:00.294301 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 18:50:00.294629 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:00.294361 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 18:50:00.448093 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:00.448060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" event={"ID":"7f630aa9-42ed-40ad-befb-a325058d93be","Type":"ContainerStarted","Data":"35b91b91e43a6c6d1d0dc0a882a3366368ca354015bc39579c2a7e1d7eca570c"} Jul 22 18:50:00.448253 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:00.448148 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:50:00.484251 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:00.484197 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" podStartSLOduration=0.81627051 podStartE2EDuration="8.484180269s" totalImagesPullingTime="7.667909759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:49:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:50:00.482235424 +0000 UTC m=+609.000046330" watchObservedRunningTime="2026-07-22 18:50:00.484180269 +0000 UTC m=+609.001991183" Jul 22 18:50:01.454949 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:01.454918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p8pwl" Jul 22 18:50:03.683772 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.683747 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:50:03.779396 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779313 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7abc826d-74de-44ee-8faa-a203322c7c9b-istiod-ca-cert\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779396 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779359 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-certs\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779396 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779384 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-data\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779413 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-podinfo\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779433 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-envoy\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779451 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-socket\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779481 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-credential-socket\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779535 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhfjq\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-kube-api-access-zhfjq\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779641 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779556 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-token\") pod \"7abc826d-74de-44ee-8faa-a203322c7c9b\" (UID: \"7abc826d-74de-44ee-8faa-a203322c7c9b\") " Jul 22 18:50:03.779973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779687 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:50:03.779973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779707 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7abc826d-74de-44ee-8faa-a203322c7c9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:50:03.779973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779866 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:50:03.779973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779889 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:50:03.779973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779935 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7abc826d-74de-44ee-8faa-a203322c7c9b-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.779973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.779963 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.780246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.780086 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:50:03.781910 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.781875 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:50:03.782041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.781885 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:50:03.782130 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.782112 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 18:50:03.782199 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.782184 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-kube-api-access-zhfjq" (OuterVolumeSpecName: "kube-api-access-zhfjq") pod "7abc826d-74de-44ee-8faa-a203322c7c9b" (UID: "7abc826d-74de-44ee-8faa-a203322c7c9b"). InnerVolumeSpecName "kube-api-access-zhfjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:50:03.881285 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881247 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.881285 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881277 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.881285 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881290 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.881521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881301 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.881521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881313 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7abc826d-74de-44ee-8faa-a203322c7c9b-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.881521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881326 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhfjq\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-kube-api-access-zhfjq\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:03.881521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:03.881339 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7abc826d-74de-44ee-8faa-a203322c7c9b-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:50:04.469670 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.469638 2566 generic.go:356] "Generic (PLEG): container finished" podID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerID="7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6" exitCode=0 Jul 22 18:50:04.469862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.469682 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" event={"ID":"7abc826d-74de-44ee-8faa-a203322c7c9b","Type":"ContainerDied","Data":"7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6"} Jul 22 18:50:04.469862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.469691 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" Jul 22 18:50:04.469862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.469723 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4" event={"ID":"7abc826d-74de-44ee-8faa-a203322c7c9b","Type":"ContainerDied","Data":"018ba28ac01b3aa4b10beead7e9598b3f7903705917dd3213fa121dd7f60413f"} Jul 22 18:50:04.469862 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.469745 2566 scope.go:117] "RemoveContainer" containerID="7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6" Jul 22 18:50:04.478360 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.478339 2566 scope.go:117] "RemoveContainer" containerID="7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6" Jul 22 18:50:04.478633 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:50:04.478614 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6\": container with ID starting with 7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6 not found: ID does not exist" containerID="7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6" Jul 22 18:50:04.478684 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.478642 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6"} err="failed to get container status \"7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6\": rpc error: code = NotFound desc = could not find container \"7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6\": container with ID starting with 7666177f09658f7a073db392e72cce8ec230dded06164279c2b2ab2d3cb342d6 not found: ID does not exist" Jul 22 18:50:04.491220 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.491190 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4"] Jul 22 18:50:04.498698 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:04.496726 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-swxw4"] Jul 22 18:50:06.043680 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:06.043642 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" path="/var/lib/kubelet/pods/7abc826d-74de-44ee-8faa-a203322c7c9b/volumes" Jul 22 18:50:54.728570 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.728536 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg"] Jul 22 18:50:54.728975 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.728917 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" Jul 22 18:50:54.728975 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.728928 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" Jul 22 18:50:54.729046 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.728987 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7abc826d-74de-44ee-8faa-a203322c7c9b" containerName="istio-proxy" Jul 22 18:50:54.733216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.733197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.736071 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.736049 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 18:50:54.736197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.736079 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8lbcw\"" Jul 22 18:50:54.736802 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.736786 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 18:50:54.740867 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.740503 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg"] Jul 22 18:50:54.813544 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.813506 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.813752 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.813577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.813752 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.813651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gqn\" (UniqueName: \"kubernetes.io/projected/36265b20-f4be-404a-a4dc-2d4bf169487d-kube-api-access-h5gqn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.914425 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.914385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.914600 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.914445 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.914600 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.914480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gqn\" (UniqueName: \"kubernetes.io/projected/36265b20-f4be-404a-a4dc-2d4bf169487d-kube-api-access-h5gqn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.914785 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.914763 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.914907 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.914872 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:54.923091 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:54.923061 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gqn\" (UniqueName: \"kubernetes.io/projected/36265b20-f4be-404a-a4dc-2d4bf169487d-kube-api-access-h5gqn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:55.043406 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.043318 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:50:55.177735 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:50:55.177707 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36265b20_f4be_404a_a4dc_2d4bf169487d.slice/crio-3ef19a5a51d5be384c8e8b0d4cc09b7b70d1ce18b7281d22a859888df963191a WatchSource:0}: Error finding container 3ef19a5a51d5be384c8e8b0d4cc09b7b70d1ce18b7281d22a859888df963191a: Status 404 returned error can't find the container with id 3ef19a5a51d5be384c8e8b0d4cc09b7b70d1ce18b7281d22a859888df963191a Jul 22 18:50:55.182548 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.182523 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg"] Jul 22 18:50:55.322594 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.322521 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v"] Jul 22 18:50:55.326050 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.326034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.334484 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.334454 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v"] Jul 22 18:50:55.419482 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.419447 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82sfs\" (UniqueName: \"kubernetes.io/projected/44643ad6-beea-4942-b96c-231e76efceb2-kube-api-access-82sfs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.419669 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.419497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.419669 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.419619 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.521143 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.521100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.521341 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.521178 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.521341 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.521260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82sfs\" (UniqueName: \"kubernetes.io/projected/44643ad6-beea-4942-b96c-231e76efceb2-kube-api-access-82sfs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.521562 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.521539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.521608 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.521545 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.530924 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.530898 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82sfs\" (UniqueName: \"kubernetes.io/projected/44643ad6-beea-4942-b96c-231e76efceb2-kube-api-access-82sfs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.651980 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.651944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:50:55.740622 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.740402 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv"] Jul 22 18:50:55.745193 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.745160 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv"] Jul 22 18:50:55.745326 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.745282 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.749047 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.748968 2566 generic.go:356] "Generic (PLEG): container finished" podID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerID="b4adaf237ccc0b2933024d3564293c9a5a8f70a49a6232c02f7d4b19ff1a167c" exitCode=0 Jul 22 18:50:55.749047 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.749033 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" event={"ID":"36265b20-f4be-404a-a4dc-2d4bf169487d","Type":"ContainerDied","Data":"b4adaf237ccc0b2933024d3564293c9a5a8f70a49a6232c02f7d4b19ff1a167c"} Jul 22 18:50:55.749235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.749074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" event={"ID":"36265b20-f4be-404a-a4dc-2d4bf169487d","Type":"ContainerStarted","Data":"3ef19a5a51d5be384c8e8b0d4cc09b7b70d1ce18b7281d22a859888df963191a"} Jul 22 18:50:55.749607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.749518 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 18:50:55.817293 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.817265 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v"] Jul 22 18:50:55.818457 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:50:55.818434 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44643ad6_beea_4942_b96c_231e76efceb2.slice/crio-1cd00f68f01528e8dad3bbd312591bd46929d79995ba14e1d6b8de5f9855c98d WatchSource:0}: Error finding container 1cd00f68f01528e8dad3bbd312591bd46929d79995ba14e1d6b8de5f9855c98d: Status 404 returned error can't find the container with id 1cd00f68f01528e8dad3bbd312591bd46929d79995ba14e1d6b8de5f9855c98d Jul 22 18:50:55.824328 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.824307 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.824423 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.824387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.824423 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.824409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8b4x\" (UniqueName: \"kubernetes.io/projected/6c686071-ac4d-4c73-a57e-75bdb39deccb-kube-api-access-b8b4x\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.925846 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.925803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.925969 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.925898 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.925969 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.925925 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8b4x\" (UniqueName: \"kubernetes.io/projected/6c686071-ac4d-4c73-a57e-75bdb39deccb-kube-api-access-b8b4x\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.926235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.926212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.926235 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.926228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:55.934333 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:55.934310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8b4x\" (UniqueName: \"kubernetes.io/projected/6c686071-ac4d-4c73-a57e-75bdb39deccb-kube-api-access-b8b4x\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:56.057429 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.057398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:50:56.127213 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.127179 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg"] Jul 22 18:50:56.131113 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.131086 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.140026 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.140001 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg"] Jul 22 18:50:56.199486 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.199455 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv"] Jul 22 18:50:56.201321 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:50:56.201294 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c686071_ac4d_4c73_a57e_75bdb39deccb.slice/crio-76d4ed61e05582a1605c0ac72d61f83bc5ff7617d9c83b1bbd2389eb723da0db WatchSource:0}: Error finding container 76d4ed61e05582a1605c0ac72d61f83bc5ff7617d9c83b1bbd2389eb723da0db: Status 404 returned error can't find the container with id 76d4ed61e05582a1605c0ac72d61f83bc5ff7617d9c83b1bbd2389eb723da0db Jul 22 18:50:56.229517 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.229492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.229634 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.229528 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpslq\" (UniqueName: \"kubernetes.io/projected/0e1d92ec-4681-4721-81f1-b9e100d16220-kube-api-access-tpslq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.229735 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.229711 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.330314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.330281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.330485 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.330323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.330485 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.330360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpslq\" (UniqueName: \"kubernetes.io/projected/0e1d92ec-4681-4721-81f1-b9e100d16220-kube-api-access-tpslq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.330677 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.330657 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.330720 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.330688 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.339514 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.339490 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpslq\" (UniqueName: \"kubernetes.io/projected/0e1d92ec-4681-4721-81f1-b9e100d16220-kube-api-access-tpslq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.442089 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.442054 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:50:56.575635 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.575600 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg"] Jul 22 18:50:56.579608 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:50:56.579572 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1d92ec_4681_4721_81f1_b9e100d16220.slice/crio-8e06c1821c008e1f03360800d7f3ef1542a152e455b6f8a950084b48988d292b WatchSource:0}: Error finding container 8e06c1821c008e1f03360800d7f3ef1542a152e455b6f8a950084b48988d292b: Status 404 returned error can't find the container with id 8e06c1821c008e1f03360800d7f3ef1542a152e455b6f8a950084b48988d292b Jul 22 18:50:56.756962 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.756865 2566 generic.go:356] "Generic (PLEG): container finished" podID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerID="eb96a7194abb89c336343057a563453f228b8e9309ba6caa61eed10483535ba1" exitCode=0 Jul 22 18:50:56.757426 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.756962 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" event={"ID":"0e1d92ec-4681-4721-81f1-b9e100d16220","Type":"ContainerDied","Data":"eb96a7194abb89c336343057a563453f228b8e9309ba6caa61eed10483535ba1"} Jul 22 18:50:56.757426 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.757022 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" event={"ID":"0e1d92ec-4681-4721-81f1-b9e100d16220","Type":"ContainerStarted","Data":"8e06c1821c008e1f03360800d7f3ef1542a152e455b6f8a950084b48988d292b"} Jul 22 18:50:56.758623 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.758601 2566 generic.go:356] "Generic (PLEG): container finished" podID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerID="01273b8663cd927b69e10a15585004cd5f5cb7ec0e0c530bf7b7ce12f2692cdb" exitCode=0 Jul 22 18:50:56.758748 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.758693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" event={"ID":"6c686071-ac4d-4c73-a57e-75bdb39deccb","Type":"ContainerDied","Data":"01273b8663cd927b69e10a15585004cd5f5cb7ec0e0c530bf7b7ce12f2692cdb"} Jul 22 18:50:56.758831 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.758763 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" event={"ID":"6c686071-ac4d-4c73-a57e-75bdb39deccb","Type":"ContainerStarted","Data":"76d4ed61e05582a1605c0ac72d61f83bc5ff7617d9c83b1bbd2389eb723da0db"} Jul 22 18:50:56.760472 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.760446 2566 generic.go:356] "Generic (PLEG): container finished" podID="44643ad6-beea-4942-b96c-231e76efceb2" containerID="d781e0417c35af16cca8740f147852c2e3557ede2704a4e57622a294f222329c" exitCode=0 Jul 22 18:50:56.760579 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.760498 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" event={"ID":"44643ad6-beea-4942-b96c-231e76efceb2","Type":"ContainerDied","Data":"d781e0417c35af16cca8740f147852c2e3557ede2704a4e57622a294f222329c"} Jul 22 18:50:56.760579 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:56.760537 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" event={"ID":"44643ad6-beea-4942-b96c-231e76efceb2","Type":"ContainerStarted","Data":"1cd00f68f01528e8dad3bbd312591bd46929d79995ba14e1d6b8de5f9855c98d"} Jul 22 18:50:58.774768 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.774725 2566 generic.go:356] "Generic (PLEG): container finished" podID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerID="54d6a6593b1dc3de00df963c2a8b1a83039e3df776c6b90ba692818b19d85ce2" exitCode=0 Jul 22 18:50:58.775202 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.774807 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" event={"ID":"6c686071-ac4d-4c73-a57e-75bdb39deccb","Type":"ContainerDied","Data":"54d6a6593b1dc3de00df963c2a8b1a83039e3df776c6b90ba692818b19d85ce2"} Jul 22 18:50:58.776837 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.776794 2566 generic.go:356] "Generic (PLEG): container finished" podID="44643ad6-beea-4942-b96c-231e76efceb2" containerID="e17add6a8064ba04769102f4b3b95adeb10c9ea61b13e8a7230605cee35b2cd3" exitCode=0 Jul 22 18:50:58.776936 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.776881 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" event={"ID":"44643ad6-beea-4942-b96c-231e76efceb2","Type":"ContainerDied","Data":"e17add6a8064ba04769102f4b3b95adeb10c9ea61b13e8a7230605cee35b2cd3"} Jul 22 18:50:58.779049 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.779026 2566 generic.go:356] "Generic (PLEG): container finished" podID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerID="b47875a89a02c9992b2843bdb9201e474d7d467acf2843b45471da19c24d7ad3" exitCode=0 Jul 22 18:50:58.779145 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.779069 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" event={"ID":"36265b20-f4be-404a-a4dc-2d4bf169487d","Type":"ContainerDied","Data":"b47875a89a02c9992b2843bdb9201e474d7d467acf2843b45471da19c24d7ad3"} Jul 22 18:50:58.780965 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.780943 2566 generic.go:356] "Generic (PLEG): container finished" podID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerID="50cce182374385cf517dcf415d5373d6ecf517a99328da55bd10e023ee394a6d" exitCode=0 Jul 22 18:50:58.781066 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:58.781010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" event={"ID":"0e1d92ec-4681-4721-81f1-b9e100d16220","Type":"ContainerDied","Data":"50cce182374385cf517dcf415d5373d6ecf517a99328da55bd10e023ee394a6d"} Jul 22 18:50:59.788214 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.788181 2566 generic.go:356] "Generic (PLEG): container finished" podID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerID="1ec985051457662d452bafcd4ca31f95cd6c4154a5f72fa0a393d206691e9241" exitCode=0 Jul 22 18:50:59.788576 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.788244 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" event={"ID":"6c686071-ac4d-4c73-a57e-75bdb39deccb","Type":"ContainerDied","Data":"1ec985051457662d452bafcd4ca31f95cd6c4154a5f72fa0a393d206691e9241"} Jul 22 18:50:59.790239 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.790221 2566 generic.go:356] "Generic (PLEG): container finished" podID="44643ad6-beea-4942-b96c-231e76efceb2" containerID="4f8a449ef61d391e379c9388d2969a5d70ab5a83497eddf6bae8a79077fcb9c3" exitCode=0 Jul 22 18:50:59.790318 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.790297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" event={"ID":"44643ad6-beea-4942-b96c-231e76efceb2","Type":"ContainerDied","Data":"4f8a449ef61d391e379c9388d2969a5d70ab5a83497eddf6bae8a79077fcb9c3"} Jul 22 18:50:59.792326 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.792305 2566 generic.go:356] "Generic (PLEG): container finished" podID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerID="efc1b48e784a276edf67b1c30676826d70c2bb87e4e1981a76558a760fcae0c6" exitCode=0 Jul 22 18:50:59.792466 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.792329 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" event={"ID":"36265b20-f4be-404a-a4dc-2d4bf169487d","Type":"ContainerDied","Data":"efc1b48e784a276edf67b1c30676826d70c2bb87e4e1981a76558a760fcae0c6"} Jul 22 18:50:59.794368 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.794347 2566 generic.go:356] "Generic (PLEG): container finished" podID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerID="e30456f9b22f96b37b99799d1415ebd92823dcc4ade87efbb37614c26c157123" exitCode=0 Jul 22 18:50:59.794453 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:50:59.794425 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" event={"ID":"0e1d92ec-4681-4721-81f1-b9e100d16220","Type":"ContainerDied","Data":"e30456f9b22f96b37b99799d1415ebd92823dcc4ade87efbb37614c26c157123"} Jul 22 18:51:00.960413 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:00.960382 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:51:01.016564 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.016539 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:51:01.028502 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.028481 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:51:01.035714 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.035699 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:51:01.077099 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.077020 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpslq\" (UniqueName: \"kubernetes.io/projected/0e1d92ec-4681-4721-81f1-b9e100d16220-kube-api-access-tpslq\") pod \"0e1d92ec-4681-4721-81f1-b9e100d16220\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " Jul 22 18:51:01.077099 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.077066 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-bundle\") pod \"0e1d92ec-4681-4721-81f1-b9e100d16220\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " Jul 22 18:51:01.077326 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.077116 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-util\") pod \"0e1d92ec-4681-4721-81f1-b9e100d16220\" (UID: \"0e1d92ec-4681-4721-81f1-b9e100d16220\") " Jul 22 18:51:01.077668 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.077638 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-bundle" (OuterVolumeSpecName: "bundle") pod "0e1d92ec-4681-4721-81f1-b9e100d16220" (UID: "0e1d92ec-4681-4721-81f1-b9e100d16220"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.079247 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.079220 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1d92ec-4681-4721-81f1-b9e100d16220-kube-api-access-tpslq" (OuterVolumeSpecName: "kube-api-access-tpslq") pod "0e1d92ec-4681-4721-81f1-b9e100d16220" (UID: "0e1d92ec-4681-4721-81f1-b9e100d16220"). InnerVolumeSpecName "kube-api-access-tpslq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:51:01.082536 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.082511 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-util" (OuterVolumeSpecName: "util") pod "0e1d92ec-4681-4721-81f1-b9e100d16220" (UID: "0e1d92ec-4681-4721-81f1-b9e100d16220"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.177758 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177721 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-util\") pod \"36265b20-f4be-404a-a4dc-2d4bf169487d\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " Jul 22 18:51:01.177931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177780 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82sfs\" (UniqueName: \"kubernetes.io/projected/44643ad6-beea-4942-b96c-231e76efceb2-kube-api-access-82sfs\") pod \"44643ad6-beea-4942-b96c-231e76efceb2\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " Jul 22 18:51:01.177931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177803 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8b4x\" (UniqueName: \"kubernetes.io/projected/6c686071-ac4d-4c73-a57e-75bdb39deccb-kube-api-access-b8b4x\") pod \"6c686071-ac4d-4c73-a57e-75bdb39deccb\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " Jul 22 18:51:01.177931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177848 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-util\") pod \"44643ad6-beea-4942-b96c-231e76efceb2\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " Jul 22 18:51:01.177931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177872 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-util\") pod \"6c686071-ac4d-4c73-a57e-75bdb39deccb\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " Jul 22 18:51:01.177931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177917 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5gqn\" (UniqueName: \"kubernetes.io/projected/36265b20-f4be-404a-a4dc-2d4bf169487d-kube-api-access-h5gqn\") pod \"36265b20-f4be-404a-a4dc-2d4bf169487d\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " Jul 22 18:51:01.178215 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.177944 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-bundle\") pod \"44643ad6-beea-4942-b96c-231e76efceb2\" (UID: \"44643ad6-beea-4942-b96c-231e76efceb2\") " Jul 22 18:51:01.178215 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178026 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-bundle\") pod \"36265b20-f4be-404a-a4dc-2d4bf169487d\" (UID: \"36265b20-f4be-404a-a4dc-2d4bf169487d\") " Jul 22 18:51:01.178215 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178056 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-bundle\") pod \"6c686071-ac4d-4c73-a57e-75bdb39deccb\" (UID: \"6c686071-ac4d-4c73-a57e-75bdb39deccb\") " Jul 22 18:51:01.178931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178414 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.178931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178437 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpslq\" (UniqueName: \"kubernetes.io/projected/0e1d92ec-4681-4721-81f1-b9e100d16220-kube-api-access-tpslq\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.178931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178451 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e1d92ec-4681-4721-81f1-b9e100d16220-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.178931 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178866 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-bundle" (OuterVolumeSpecName: "bundle") pod "44643ad6-beea-4942-b96c-231e76efceb2" (UID: "44643ad6-beea-4942-b96c-231e76efceb2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.179363 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.178966 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-bundle" (OuterVolumeSpecName: "bundle") pod "6c686071-ac4d-4c73-a57e-75bdb39deccb" (UID: "6c686071-ac4d-4c73-a57e-75bdb39deccb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.179802 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.179772 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-bundle" (OuterVolumeSpecName: "bundle") pod "36265b20-f4be-404a-a4dc-2d4bf169487d" (UID: "36265b20-f4be-404a-a4dc-2d4bf169487d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.180607 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.180580 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44643ad6-beea-4942-b96c-231e76efceb2-kube-api-access-82sfs" (OuterVolumeSpecName: "kube-api-access-82sfs") pod "44643ad6-beea-4942-b96c-231e76efceb2" (UID: "44643ad6-beea-4942-b96c-231e76efceb2"). InnerVolumeSpecName "kube-api-access-82sfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:51:01.180710 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.180607 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c686071-ac4d-4c73-a57e-75bdb39deccb-kube-api-access-b8b4x" (OuterVolumeSpecName: "kube-api-access-b8b4x") pod "6c686071-ac4d-4c73-a57e-75bdb39deccb" (UID: "6c686071-ac4d-4c73-a57e-75bdb39deccb"). InnerVolumeSpecName "kube-api-access-b8b4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:51:01.181311 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.181280 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36265b20-f4be-404a-a4dc-2d4bf169487d-kube-api-access-h5gqn" (OuterVolumeSpecName: "kube-api-access-h5gqn") pod "36265b20-f4be-404a-a4dc-2d4bf169487d" (UID: "36265b20-f4be-404a-a4dc-2d4bf169487d"). InnerVolumeSpecName "kube-api-access-h5gqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:51:01.183501 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.183477 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-util" (OuterVolumeSpecName: "util") pod "44643ad6-beea-4942-b96c-231e76efceb2" (UID: "44643ad6-beea-4942-b96c-231e76efceb2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.186200 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.186174 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-util" (OuterVolumeSpecName: "util") pod "6c686071-ac4d-4c73-a57e-75bdb39deccb" (UID: "6c686071-ac4d-4c73-a57e-75bdb39deccb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.186439 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.186422 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-util" (OuterVolumeSpecName: "util") pod "36265b20-f4be-404a-a4dc-2d4bf169487d" (UID: "36265b20-f4be-404a-a4dc-2d4bf169487d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 18:51:01.279582 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279546 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279582 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279574 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279582 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279583 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36265b20-f4be-404a-a4dc-2d4bf169487d-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279582 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279591 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82sfs\" (UniqueName: \"kubernetes.io/projected/44643ad6-beea-4942-b96c-231e76efceb2-kube-api-access-82sfs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279600 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8b4x\" (UniqueName: \"kubernetes.io/projected/6c686071-ac4d-4c73-a57e-75bdb39deccb-kube-api-access-b8b4x\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279609 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279617 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6c686071-ac4d-4c73-a57e-75bdb39deccb-util\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279626 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5gqn\" (UniqueName: \"kubernetes.io/projected/36265b20-f4be-404a-a4dc-2d4bf169487d-kube-api-access-h5gqn\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.279864 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.279634 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44643ad6-beea-4942-b96c-231e76efceb2-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:51:01.807173 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.807143 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" Jul 22 18:51:01.807335 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.807135 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv" event={"ID":"6c686071-ac4d-4c73-a57e-75bdb39deccb","Type":"ContainerDied","Data":"76d4ed61e05582a1605c0ac72d61f83bc5ff7617d9c83b1bbd2389eb723da0db"} Jul 22 18:51:01.807335 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.807314 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76d4ed61e05582a1605c0ac72d61f83bc5ff7617d9c83b1bbd2389eb723da0db" Jul 22 18:51:01.809273 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.809250 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" Jul 22 18:51:01.809273 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.809262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v" event={"ID":"44643ad6-beea-4942-b96c-231e76efceb2","Type":"ContainerDied","Data":"1cd00f68f01528e8dad3bbd312591bd46929d79995ba14e1d6b8de5f9855c98d"} Jul 22 18:51:01.809468 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.809315 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1cd00f68f01528e8dad3bbd312591bd46929d79995ba14e1d6b8de5f9855c98d" Jul 22 18:51:01.811260 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.811235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" event={"ID":"36265b20-f4be-404a-a4dc-2d4bf169487d","Type":"ContainerDied","Data":"3ef19a5a51d5be384c8e8b0d4cc09b7b70d1ce18b7281d22a859888df963191a"} Jul 22 18:51:01.811370 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.811278 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ef19a5a51d5be384c8e8b0d4cc09b7b70d1ce18b7281d22a859888df963191a" Jul 22 18:51:01.811370 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.811243 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg" Jul 22 18:51:01.813154 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.813139 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" Jul 22 18:51:01.813154 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.813140 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg" event={"ID":"0e1d92ec-4681-4721-81f1-b9e100d16220","Type":"ContainerDied","Data":"8e06c1821c008e1f03360800d7f3ef1542a152e455b6f8a950084b48988d292b"} Jul 22 18:51:01.813280 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:01.813180 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e06c1821c008e1f03360800d7f3ef1542a152e455b6f8a950084b48988d292b" Jul 22 18:51:46.052507 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.052416 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-557bb65c57-kc7r7"] Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.052943 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="util" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.052964 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="util" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.052978 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="pull" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.052989 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="pull" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053003 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="util" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053012 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="util" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053027 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="extract" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053037 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="extract" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053062 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="extract" Jul 22 18:51:46.053069 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053071 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053084 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="util" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053094 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="util" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053105 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="util" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053113 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="util" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053124 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="pull" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053134 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="pull" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053143 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="pull" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053151 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="pull" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053165 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053174 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053184 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="pull" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053192 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="pull" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053204 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053212 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053297 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e1d92ec-4681-4721-81f1-b9e100d16220" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053312 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36265b20-f4be-404a-a4dc-2d4bf169487d" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053323 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="44643ad6-beea-4942-b96c-231e76efceb2" containerName="extract" Jul 22 18:51:46.053609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.053333 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c686071-ac4d-4c73-a57e-75bdb39deccb" containerName="extract" Jul 22 18:51:46.056329 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.056307 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.069898 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.069871 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-557bb65c57-kc7r7"] Jul 22 18:51:46.176144 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176106 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-trusted-ca-bundle\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.176313 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176155 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv2tm\" (UniqueName: \"kubernetes.io/projected/4d65bb9c-0bcb-45f9-af67-114b623a7b49-kube-api-access-hv2tm\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.176313 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176212 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-oauth-serving-cert\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.176313 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-service-ca\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.176313 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176290 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-config\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.176454 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-oauth-config\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.176454 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.176368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-serving-cert\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277302 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-trusted-ca-bundle\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv2tm\" (UniqueName: \"kubernetes.io/projected/4d65bb9c-0bcb-45f9-af67-114b623a7b49-kube-api-access-hv2tm\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277341 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-oauth-serving-cert\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-service-ca\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277509 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277426 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-config\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277722 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277505 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-oauth-config\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.277722 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.277571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-serving-cert\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.278260 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.278235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-oauth-serving-cert\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.278392 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.278269 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-config\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.278464 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.278388 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-service-ca\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.278464 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.278452 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4d65bb9c-0bcb-45f9-af67-114b623a7b49-trusted-ca-bundle\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.280033 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.280005 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-serving-cert\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.280117 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.280102 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4d65bb9c-0bcb-45f9-af67-114b623a7b49-console-oauth-config\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.285057 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.285035 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv2tm\" (UniqueName: \"kubernetes.io/projected/4d65bb9c-0bcb-45f9-af67-114b623a7b49-kube-api-access-hv2tm\") pod \"console-557bb65c57-kc7r7\" (UID: \"4d65bb9c-0bcb-45f9-af67-114b623a7b49\") " pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.366728 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.366696 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:46.504330 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:46.504301 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-557bb65c57-kc7r7"] Jul 22 18:51:46.507118 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:51:46.507077 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d65bb9c_0bcb_45f9_af67_114b623a7b49.slice/crio-bf52387dd15a4e3a0a2a3ada7810e1ffa81b53b3f48336806d8663f04eadef3c WatchSource:0}: Error finding container bf52387dd15a4e3a0a2a3ada7810e1ffa81b53b3f48336806d8663f04eadef3c: Status 404 returned error can't find the container with id bf52387dd15a4e3a0a2a3ada7810e1ffa81b53b3f48336806d8663f04eadef3c Jul 22 18:51:47.048260 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:47.048214 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-557bb65c57-kc7r7" event={"ID":"4d65bb9c-0bcb-45f9-af67-114b623a7b49","Type":"ContainerStarted","Data":"d69e69aa9896ca6fc6a3b065b445c34832bb755f89b0c99365a9e59f9c1d6603"} Jul 22 18:51:47.048260 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:47.048265 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-557bb65c57-kc7r7" event={"ID":"4d65bb9c-0bcb-45f9-af67-114b623a7b49","Type":"ContainerStarted","Data":"bf52387dd15a4e3a0a2a3ada7810e1ffa81b53b3f48336806d8663f04eadef3c"} Jul 22 18:51:47.068656 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:47.068602 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-557bb65c57-kc7r7" podStartSLOduration=1.068585954 podStartE2EDuration="1.068585954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:51:47.066802127 +0000 UTC m=+715.584613040" watchObservedRunningTime="2026-07-22 18:51:47.068585954 +0000 UTC m=+715.586396867" Jul 22 18:51:56.366908 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:56.366872 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:56.367393 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:56.366934 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:56.371651 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:56.371628 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:57.110665 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:57.110636 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-557bb65c57-kc7r7" Jul 22 18:51:57.158180 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:51:57.158141 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64474db96-68672"] Jul 22 18:52:22.183301 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.183261 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64474db96-68672" podUID="fece5d9a-0c11-49e3-a9cd-2310e44df1c0" containerName="console" containerID="cri-o://0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f" gracePeriod=15 Jul 22 18:52:22.433403 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.433341 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64474db96-68672" Jul 22 18:52:22.505861 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.505809 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-config\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506021 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.505893 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-service-ca\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506021 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.505985 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-serving-cert\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506041 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-oauth-serving-cert\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506100 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506071 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-trusted-ca-bundle\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506179 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506136 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhwhs\" (UniqueName: \"kubernetes.io/projected/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-kube-api-access-lhwhs\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506179 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506164 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-oauth-config\") pod \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\" (UID: \"fece5d9a-0c11-49e3-a9cd-2310e44df1c0\") " Jul 22 18:52:22.506271 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506241 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-service-ca" (OuterVolumeSpecName: "service-ca") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:52:22.506332 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506238 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-config" (OuterVolumeSpecName: "console-config") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:52:22.506484 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506458 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:52:22.506612 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506521 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 18:52:22.506612 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506574 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-service-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:22.506612 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506592 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-oauth-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:22.506612 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.506602 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:22.508224 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.508200 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:52:22.508330 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.508269 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-kube-api-access-lhwhs" (OuterVolumeSpecName: "kube-api-access-lhwhs") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "kube-api-access-lhwhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 18:52:22.508330 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.508303 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fece5d9a-0c11-49e3-a9cd-2310e44df1c0" (UID: "fece5d9a-0c11-49e3-a9cd-2310e44df1c0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 18:52:22.607941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.607890 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhwhs\" (UniqueName: \"kubernetes.io/projected/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-kube-api-access-lhwhs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:22.607941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.607932 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-oauth-config\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:22.607941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.607941 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-console-serving-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:22.607941 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:22.607951 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fece5d9a-0c11-49e3-a9cd-2310e44df1c0-trusted-ca-bundle\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 18:52:23.245524 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.245488 2566 generic.go:356] "Generic (PLEG): container finished" podID="fece5d9a-0c11-49e3-a9cd-2310e44df1c0" containerID="0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f" exitCode=0 Jul 22 18:52:23.245986 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.245534 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64474db96-68672" Jul 22 18:52:23.245986 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.245574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64474db96-68672" event={"ID":"fece5d9a-0c11-49e3-a9cd-2310e44df1c0","Type":"ContainerDied","Data":"0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f"} Jul 22 18:52:23.245986 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.245631 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64474db96-68672" event={"ID":"fece5d9a-0c11-49e3-a9cd-2310e44df1c0","Type":"ContainerDied","Data":"c060bc256e8f893b7522fd8ccd02412259cd6ec514bf29647bfd039e6ba94f65"} Jul 22 18:52:23.245986 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.245654 2566 scope.go:117] "RemoveContainer" containerID="0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f" Jul 22 18:52:23.255663 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.255646 2566 scope.go:117] "RemoveContainer" containerID="0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f" Jul 22 18:52:23.255955 ip-10-0-137-148 kubenswrapper[2566]: E0722 18:52:23.255935 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f\": container with ID starting with 0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f not found: ID does not exist" containerID="0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f" Jul 22 18:52:23.256033 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.255968 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f"} err="failed to get container status \"0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f\": rpc error: code = NotFound desc = could not find container \"0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f\": container with ID starting with 0290c2b5526c3a62ce45a0dd2a47001b168c0b1d23e81436cd74d8fece54457f not found: ID does not exist" Jul 22 18:52:23.267759 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.267737 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64474db96-68672"] Jul 22 18:52:23.270690 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:23.270670 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64474db96-68672"] Jul 22 18:52:24.043585 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:24.043555 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fece5d9a-0c11-49e3-a9cd-2310e44df1c0" path="/var/lib/kubelet/pods/fece5d9a-0c11-49e3-a9cd-2310e44df1c0/volumes" Jul 22 18:52:35.939775 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.939739 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:52:35.940246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.940108 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fece5d9a-0c11-49e3-a9cd-2310e44df1c0" containerName="console" Jul 22 18:52:35.940246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.940119 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fece5d9a-0c11-49e3-a9cd-2310e44df1c0" containerName="console" Jul 22 18:52:35.940246 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.940190 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fece5d9a-0c11-49e3-a9cd-2310e44df1c0" containerName="console" Jul 22 18:52:35.942927 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.942904 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:35.945411 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.945389 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 18:52:35.945523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.945431 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 18:52:35.945523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.945394 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8lbcw\"" Jul 22 18:52:35.945523 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.945394 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 18:52:35.952724 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.952694 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:52:35.982728 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:35.982698 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:52:36.027439 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.027403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gmcx\" (UniqueName: \"kubernetes.io/projected/8b15d19d-74e8-427c-9802-3c4b252f32dd-kube-api-access-8gmcx\") pod \"limitador-limitador-54d6d988d5-54bq2\" (UID: \"8b15d19d-74e8-427c-9802-3c4b252f32dd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.027608 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.027484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b15d19d-74e8-427c-9802-3c4b252f32dd-config-file\") pod \"limitador-limitador-54d6d988d5-54bq2\" (UID: \"8b15d19d-74e8-427c-9802-3c4b252f32dd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.128546 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.128504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gmcx\" (UniqueName: \"kubernetes.io/projected/8b15d19d-74e8-427c-9802-3c4b252f32dd-kube-api-access-8gmcx\") pod \"limitador-limitador-54d6d988d5-54bq2\" (UID: \"8b15d19d-74e8-427c-9802-3c4b252f32dd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.128774 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.128732 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b15d19d-74e8-427c-9802-3c4b252f32dd-config-file\") pod \"limitador-limitador-54d6d988d5-54bq2\" (UID: \"8b15d19d-74e8-427c-9802-3c4b252f32dd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.129314 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.129294 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8b15d19d-74e8-427c-9802-3c4b252f32dd-config-file\") pod \"limitador-limitador-54d6d988d5-54bq2\" (UID: \"8b15d19d-74e8-427c-9802-3c4b252f32dd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.136917 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.136889 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gmcx\" (UniqueName: \"kubernetes.io/projected/8b15d19d-74e8-427c-9802-3c4b252f32dd-kube-api-access-8gmcx\") pod \"limitador-limitador-54d6d988d5-54bq2\" (UID: \"8b15d19d-74e8-427c-9802-3c4b252f32dd\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.258807 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.258690 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:36.391926 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:36.391902 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:52:36.393283 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:52:36.393248 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b15d19d_74e8_427c_9802_3c4b252f32dd.slice/crio-3d013a9e74c7a6204f53156c0a83eaa969cc290b7e338d9e3f221e897cfd7503 WatchSource:0}: Error finding container 3d013a9e74c7a6204f53156c0a83eaa969cc290b7e338d9e3f221e897cfd7503: Status 404 returned error can't find the container with id 3d013a9e74c7a6204f53156c0a83eaa969cc290b7e338d9e3f221e897cfd7503 Jul 22 18:52:37.322664 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:37.322625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" event={"ID":"8b15d19d-74e8-427c-9802-3c4b252f32dd","Type":"ContainerStarted","Data":"3d013a9e74c7a6204f53156c0a83eaa969cc290b7e338d9e3f221e897cfd7503"} Jul 22 18:52:39.335744 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:39.335651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" event={"ID":"8b15d19d-74e8-427c-9802-3c4b252f32dd","Type":"ContainerStarted","Data":"d2051d81b3d61b5d5b7c1989c28194f1052bfe9c06eaf7d7a6bd9c4dc9ae64b2"} Jul 22 18:52:39.336108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:39.335778 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:52:39.352969 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:39.352918 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" podStartSLOduration=1.686233166 podStartE2EDuration="4.352900916s" totalImagesPullingTime="2.66666775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:52:39.352301516 +0000 UTC m=+767.870112455" watchObservedRunningTime="2026-07-22 18:52:39.352900916 +0000 UTC m=+767.870711852" Jul 22 18:52:50.342176 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:52:50.342145 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-54bq2" Jul 22 18:53:47.183638 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:53:47.183605 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:54:04.888229 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:04.888190 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:54:12.694330 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:12.694297 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:54:22.189858 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.189808 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc"] Jul 22 18:54:22.193645 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.193624 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.195747 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.195725 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 18:54:22.196421 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.196399 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 18:54:22.196553 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.196532 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-92cn2\"" Jul 22 18:54:22.196640 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.196623 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 22 18:54:22.201253 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.201232 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc"] Jul 22 18:54:22.286609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.286571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29vqb\" (UniqueName: \"kubernetes.io/projected/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-kube-api-access-29vqb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.286609 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.286625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.286932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.286645 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.286932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.286713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.286932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.286848 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.286932 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.286871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.387604 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.387565 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.387604 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.387606 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.387866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.387634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.387866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.387723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.387866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.387753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.387866 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.387786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29vqb\" (UniqueName: \"kubernetes.io/projected/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-kube-api-access-29vqb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.388084 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.388051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.388145 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.388090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.388201 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.388170 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.389986 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.389965 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.390149 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.390134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.395521 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.395499 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29vqb\" (UniqueName: \"kubernetes.io/projected/baa82d72-e50f-476f-b8d8-9a8ce1cd6976-kube-api-access-29vqb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8zhhc\" (UID: \"baa82d72-e50f-476f-b8d8-9a8ce1cd6976\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.506870 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.506765 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:22.636013 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.635988 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc"] Jul 22 18:54:22.638209 ip-10-0-137-148 kubenswrapper[2566]: W0722 18:54:22.638179 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa82d72_e50f_476f_b8d8_9a8ce1cd6976.slice/crio-f71c4fd2a8ff439e36986af358c7f6c9421645ef5be089b6934a1d7ebbfc3c93 WatchSource:0}: Error finding container f71c4fd2a8ff439e36986af358c7f6c9421645ef5be089b6934a1d7ebbfc3c93: Status 404 returned error can't find the container with id f71c4fd2a8ff439e36986af358c7f6c9421645ef5be089b6934a1d7ebbfc3c93 Jul 22 18:54:22.900840 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:22.900759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" event={"ID":"baa82d72-e50f-476f-b8d8-9a8ce1cd6976","Type":"ContainerStarted","Data":"f71c4fd2a8ff439e36986af358c7f6c9421645ef5be089b6934a1d7ebbfc3c93"} Jul 22 18:54:23.492283 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:23.492246 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:54:23.907971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:23.907934 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" event={"ID":"baa82d72-e50f-476f-b8d8-9a8ce1cd6976","Type":"ContainerStarted","Data":"f5ad31adbecbc32f5efebe0f994c7d5d57b14301042c1f73b409c120dc3c09a0"} Jul 22 18:54:23.926989 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:23.926932 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" podStartSLOduration=0.912547071 podStartE2EDuration="1.926913216s" totalImagesPullingTime="1.014366145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 18:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 18:54:23.924945956 +0000 UTC m=+872.442756873" watchObservedRunningTime="2026-07-22 18:54:23.926913216 +0000 UTC m=+872.444724129" Jul 22 18:54:32.507798 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:32.507754 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:32.508268 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:32.507845 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:32.520811 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:32.520776 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:32.967332 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:32.967303 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8zhhc" Jul 22 18:54:45.685110 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:45.685031 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:54:52.006791 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:52.006761 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:54:52.009868 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:52.009845 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:54:52.013188 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:52.013164 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:54:52.015662 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:52.015632 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:54:59.184197 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:54:59.184164 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:55:10.785216 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:55:10.785172 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:56:43.588973 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:56:43.588932 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:56:54.886292 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:56:54.886250 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:57:05.388005 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:57:05.387970 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:58:43.593487 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:58:43.593451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:58:52.784108 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:58:52.784068 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:59:25.985835 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:25.985733 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:59:35.291378 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:35.291343 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:59:42.987441 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:42.987403 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:59:50.393041 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:50.392998 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 18:59:52.045971 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:52.045943 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:59:52.050852 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:52.050811 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 18:59:52.052310 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:52.052290 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:59:52.056463 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:52.056448 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 18:59:59.589446 ip-10-0-137-148 kubenswrapper[2566]: I0722 18:59:59.589401 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:00:00.149323 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.149285 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8"] Jul 22 19:00:00.152830 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.152803 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" Jul 22 19:00:00.155010 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.154993 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\"" Jul 22 19:00:00.159356 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.159324 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8"] Jul 22 19:00:00.164271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.164253 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" Jul 22 19:00:00.309150 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.309124 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8"] Jul 22 19:00:00.310963 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:00:00.310931 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b86eb9_93ac_4535_8f10_577fe804110e.slice/crio-e952f20b90d516e7efa6cce045a511c9ebaf5e6146783f55ae28452c8b2fc208 WatchSource:0}: Error finding container e952f20b90d516e7efa6cce045a511c9ebaf5e6146783f55ae28452c8b2fc208: Status 404 returned error can't find the container with id e952f20b90d516e7efa6cce045a511c9ebaf5e6146783f55ae28452c8b2fc208 Jul 22 19:00:00.312972 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.312950 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:00:00.688354 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:00.688317 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" event={"ID":"12b86eb9-93ac-4535-8f10-577fe804110e","Type":"ContainerStarted","Data":"e952f20b90d516e7efa6cce045a511c9ebaf5e6146783f55ae28452c8b2fc208"} Jul 22 19:00:03.710393 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:03.710357 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" event={"ID":"12b86eb9-93ac-4535-8f10-577fe804110e","Type":"ContainerStarted","Data":"c685e12a71490b4c8e6d2fdf4d641b722f2ca425eca318ad49f21130185c11ce"} Jul 22 19:00:03.724527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:03.724459 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" podStartSLOduration=1.073419873 podStartE2EDuration="3.72442772s" totalImagesPullingTime="2.651007847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:00:03.723026745 +0000 UTC m=+1212.240837660" watchObservedRunningTime="2026-07-22 19:00:03.72442772 +0000 UTC m=+1212.242238642" Jul 22 19:00:04.716943 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:04.716912 2566 generic.go:356] "Generic (PLEG): container finished" podID="12b86eb9-93ac-4535-8f10-577fe804110e" containerID="c685e12a71490b4c8e6d2fdf4d641b722f2ca425eca318ad49f21130185c11ce" exitCode=0 Jul 22 19:00:04.717329 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:04.716950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" event={"ID":"12b86eb9-93ac-4535-8f10-577fe804110e","Type":"ContainerDied","Data":"c685e12a71490b4c8e6d2fdf4d641b722f2ca425eca318ad49f21130185c11ce"} Jul 22 19:00:05.861743 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:05.861715 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" Jul 22 19:00:06.729536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:06.729497 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" event={"ID":"12b86eb9-93ac-4535-8f10-577fe804110e","Type":"ContainerDied","Data":"e952f20b90d516e7efa6cce045a511c9ebaf5e6146783f55ae28452c8b2fc208"} Jul 22 19:00:06.729715 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:06.729546 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e952f20b90d516e7efa6cce045a511c9ebaf5e6146783f55ae28452c8b2fc208" Jul 22 19:00:06.729715 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:06.729512 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29745780-wjqh8" Jul 22 19:00:10.092190 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:10.092150 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:00:22.493102 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:22.493064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:00:31.892589 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:31.892550 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:00:56.994117 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:00:56.994035 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:01:06.698432 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:01:06.698389 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:01:29.296282 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:01:29.296246 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:01:40.188933 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:01:40.188901 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:01:56.892395 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:01:56.892355 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:02:07.087308 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:02:07.087272 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:02:16.796058 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:02:16.795979 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:02:26.596196 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:02:26.596161 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:02:36.395801 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:02:36.395769 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:04:24.083857 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:24.083810 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:04:33.087885 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:33.087849 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:04:46.883279 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:46.883236 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:04:52.082244 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:52.082215 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:04:52.087504 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:52.087481 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:04:52.089010 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:52.088990 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:04:52.094786 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:52.094769 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:04:56.894103 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:04:56.894069 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:05:08.788535 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:05:08.788503 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:05:18.394310 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:05:18.394228 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:05:28.890340 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:05:28.890308 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:05:38.582439 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:05:38.582400 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:05:49.490271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:05:49.490232 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:05:57.193082 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:05:57.193039 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:06:06.698012 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:06:06.697981 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:06:14.493019 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:06:14.492981 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:06:33.486228 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:06:33.486193 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:06:43.083362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:06:43.083309 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:06:58.489111 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:06:58.489024 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:07:13.595198 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:07:13.595151 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:07:49.688112 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:07:49.688072 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:08:04.187886 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:08:04.187847 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:08:11.308559 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:08:11.308525 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:08:26.700311 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:08:26.700219 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:08:36.489546 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:08:36.489510 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:08:46.285538 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:08:46.285504 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:08:54.791448 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:08:54.791413 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:09:05.586743 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:05.586694 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:09:15.095762 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:15.095727 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:09:28.289936 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:28.289904 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:09:37.282367 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:37.282334 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:09:47.092907 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:47.092832 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:09:52.120432 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:52.120408 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:09:52.126292 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:52.126266 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:09:52.135175 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:52.135147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:09:52.140801 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:52.140783 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:09:56.889165 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:09:56.889131 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:10:09.914078 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:10:09.914038 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:10:19.299027 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:10:19.298984 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:10:30.693078 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:10:30.693043 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:10:42.086771 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:10:42.086722 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:10:51.192282 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:10:51.192241 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:10:59.387929 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:10:59.387891 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:11:08.592573 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:08.592535 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:11:30.606804 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:30.606704 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:11:36.200286 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.200246 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x"] Jul 22 19:11:36.200734 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.200626 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12b86eb9-93ac-4535-8f10-577fe804110e" containerName="cleanup" Jul 22 19:11:36.200734 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.200639 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="12b86eb9-93ac-4535-8f10-577fe804110e" containerName="cleanup" Jul 22 19:11:36.200734 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.200716 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="12b86eb9-93ac-4535-8f10-577fe804110e" containerName="cleanup" Jul 22 19:11:36.203677 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.203660 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.206335 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.206315 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-3c49b710-openshift-default-dockercfg-4jvn8\"" Jul 22 19:11:36.224236 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.224205 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x"] Jul 22 19:11:36.265118 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265087 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-certs\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265118 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-envoy\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265352 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-istio-token\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265352 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265243 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-socket\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265352 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-data\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265352 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265310 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a308f301-611e-4e73-9436-86ea4e354e6e-istiod-ca-cert\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265352 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265347 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a308f301-611e-4e73-9436-86ea4e354e6e-istio-podinfo\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265517 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-credential-socket\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.265517 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.265498 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x5tq\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-kube-api-access-9x5tq\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366240 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x5tq\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-kube-api-access-9x5tq\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366240 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-certs\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366508 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-envoy\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366508 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366303 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-istio-token\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366508 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-socket\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366508 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-data\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366708 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a308f301-611e-4e73-9436-86ea4e354e6e-istiod-ca-cert\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366708 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a308f301-611e-4e73-9436-86ea4e354e6e-istio-podinfo\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366708 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-credential-socket\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366933 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366869 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-data\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.366933 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.366902 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-socket\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.367049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.367033 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-certs\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.367102 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.367041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-credential-socket\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.367284 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.367257 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a308f301-611e-4e73-9436-86ea4e354e6e-istiod-ca-cert\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.368903 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.368880 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-envoy\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.369301 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.369283 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a308f301-611e-4e73-9436-86ea4e354e6e-istio-podinfo\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.374087 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.374058 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-istio-token\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.374247 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.374224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x5tq\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-kube-api-access-9x5tq\") pod \"e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.513503 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.513423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:36.654870 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.654830 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x"] Jul 22 19:11:36.662106 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:11:36.662072 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda308f301_611e_4e73_9436_86ea4e354e6e.slice/crio-6b65e10252d7ab60bf89399401c6c7d4bbef3cce7fc1f13ff2e9634634a622c2 WatchSource:0}: Error finding container 6b65e10252d7ab60bf89399401c6c7d4bbef3cce7fc1f13ff2e9634634a622c2: Status 404 returned error can't find the container with id 6b65e10252d7ab60bf89399401c6c7d4bbef3cce7fc1f13ff2e9634634a622c2 Jul 22 19:11:36.664281 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.664244 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:11:36.664384 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.664310 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:11:36.664384 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:36.664338 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:11:37.451952 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:37.451906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" event={"ID":"a308f301-611e-4e73-9436-86ea4e354e6e","Type":"ContainerStarted","Data":"5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb"} Jul 22 19:11:37.452437 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:37.451979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" event={"ID":"a308f301-611e-4e73-9436-86ea4e354e6e","Type":"ContainerStarted","Data":"6b65e10252d7ab60bf89399401c6c7d4bbef3cce7fc1f13ff2e9634634a622c2"} Jul 22 19:11:37.477720 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:37.477671 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podStartSLOduration=1.477656742 podStartE2EDuration="1.477656742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:11:37.475391395 +0000 UTC m=+1905.993202309" watchObservedRunningTime="2026-07-22 19:11:37.477656742 +0000 UTC m=+1905.995467656" Jul 22 19:11:37.513954 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:37.513925 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:37.518970 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:37.518942 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:38.456888 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:38.456855 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:38.457879 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:38.457861 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:45.696832 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:45.696793 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:11:54.132797 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.132126 2566 patch_prober.go:28] interesting pod/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": read tcp 10.133.0.2:51740->10.133.0.46:15021: read: connection reset by peer" start-of-body= Jul 22 19:11:54.132797 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.132204 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": read tcp 10.133.0.2:51740->10.133.0.46:15021: read: connection reset by peer" Jul 22 19:11:54.548978 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.548885 2566 generic.go:356] "Generic (PLEG): container finished" podID="a308f301-611e-4e73-9436-86ea4e354e6e" containerID="5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb" exitCode=0 Jul 22 19:11:54.548978 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.548959 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" event={"ID":"a308f301-611e-4e73-9436-86ea4e354e6e","Type":"ContainerDied","Data":"5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb"} Jul 22 19:11:54.549386 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.549371 2566 scope.go:117] "RemoveContainer" containerID="5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb" Jul 22 19:11:54.550366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.550336 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:11:54.550421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.550408 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:11:54.550462 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:54.550443 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:11:55.558044 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:55.558011 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" event={"ID":"a308f301-611e-4e73-9436-86ea4e354e6e","Type":"ContainerStarted","Data":"433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd"} Jul 22 19:11:56.513921 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:56.513877 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:11:56.514234 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:56.514127 2566 patch_prober.go:28] interesting pod/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 19:11:56.514422 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:56.514396 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 19:11:57.514624 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:57.514583 2566 patch_prober.go:28] interesting pod/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 19:11:57.515268 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:57.515238 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 19:11:58.514636 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:58.514596 2566 patch_prober.go:28] interesting pod/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 19:11:58.515258 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:58.515229 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 19:11:59.513787 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:59.513747 2566 patch_prober.go:28] interesting pod/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 19:11:59.514031 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:11:59.513805 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 19:12:00.517859 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:00.517810 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:12:00.518270 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:00.518103 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:12:00.518733 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:00.518708 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:12:04.511507 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.511471 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf"] Jul 22 19:12:04.513995 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.513977 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:04.516254 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.516233 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 19:12:04.516372 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.516305 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r5ttk\"" Jul 22 19:12:04.522941 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.522916 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf"] Jul 22 19:12:04.621967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.621930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccff4267-c5cd-4067-8de2-174870d1ed01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf\" (UID: \"ccff4267-c5cd-4067-8de2-174870d1ed01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:04.723291 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.723255 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccff4267-c5cd-4067-8de2-174870d1ed01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf\" (UID: \"ccff4267-c5cd-4067-8de2-174870d1ed01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:04.724008 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.723981 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccff4267-c5cd-4067-8de2-174870d1ed01-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf\" (UID: \"ccff4267-c5cd-4067-8de2-174870d1ed01\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:04.827034 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.826944 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:04.960126 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:04.960082 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf"] Jul 22 19:12:04.962510 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:12:04.962484 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccff4267_c5cd_4067_8de2_174870d1ed01.slice/crio-4f6631619fbd8b2c9244ef1aae76d0dc050e7c9bd6f774a20734b62ccc15300b WatchSource:0}: Error finding container 4f6631619fbd8b2c9244ef1aae76d0dc050e7c9bd6f774a20734b62ccc15300b: Status 404 returned error can't find the container with id 4f6631619fbd8b2c9244ef1aae76d0dc050e7c9bd6f774a20734b62ccc15300b Jul 22 19:12:05.614469 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:05.614434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" event={"ID":"ccff4267-c5cd-4067-8de2-174870d1ed01","Type":"ContainerStarted","Data":"48c3e82c3a3e3ccde0a3cec1b4b7852c7bfc6b305853950394b103941a107739"} Jul 22 19:12:05.614926 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:05.614486 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" event={"ID":"ccff4267-c5cd-4067-8de2-174870d1ed01","Type":"ContainerStarted","Data":"4f6631619fbd8b2c9244ef1aae76d0dc050e7c9bd6f774a20734b62ccc15300b"} Jul 22 19:12:05.630030 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:05.629889 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" podStartSLOduration=1.629866828 podStartE2EDuration="1.629866828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:12:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:12:05.628276821 +0000 UTC m=+1934.146087734" watchObservedRunningTime="2026-07-22 19:12:05.629866828 +0000 UTC m=+1934.147677743" Jul 22 19:12:06.622127 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:06.622096 2566 generic.go:356] "Generic (PLEG): container finished" podID="ccff4267-c5cd-4067-8de2-174870d1ed01" containerID="48c3e82c3a3e3ccde0a3cec1b4b7852c7bfc6b305853950394b103941a107739" exitCode=0 Jul 22 19:12:06.622488 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:06.622167 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" event={"ID":"ccff4267-c5cd-4067-8de2-174870d1ed01","Type":"ContainerDied","Data":"48c3e82c3a3e3ccde0a3cec1b4b7852c7bfc6b305853950394b103941a107739"} Jul 22 19:12:07.761844 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:07.761806 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:07.856149 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:07.856114 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccff4267-c5cd-4067-8de2-174870d1ed01-maas-api-service-ca\") pod \"ccff4267-c5cd-4067-8de2-174870d1ed01\" (UID: \"ccff4267-c5cd-4067-8de2-174870d1ed01\") " Jul 22 19:12:07.856470 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:07.856446 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccff4267-c5cd-4067-8de2-174870d1ed01-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ccff4267-c5cd-4067-8de2-174870d1ed01" (UID: "ccff4267-c5cd-4067-8de2-174870d1ed01"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:12:07.956959 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:07.956876 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ccff4267-c5cd-4067-8de2-174870d1ed01-maas-api-service-ca\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:08.633974 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:08.633945 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" Jul 22 19:12:08.634147 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:08.633943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf" event={"ID":"ccff4267-c5cd-4067-8de2-174870d1ed01","Type":"ContainerDied","Data":"4f6631619fbd8b2c9244ef1aae76d0dc050e7c9bd6f774a20734b62ccc15300b"} Jul 22 19:12:08.634147 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:08.634077 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f6631619fbd8b2c9244ef1aae76d0dc050e7c9bd6f774a20734b62ccc15300b" Jul 22 19:12:18.090789 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:18.090751 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:12:19.594016 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:19.593979 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf"] Jul 22 19:12:19.596252 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:19.596229 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3c49b710-q2zhf"] Jul 22 19:12:19.733102 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:19.733067 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x"] Jul 22 19:12:19.733375 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:19.733348 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" containerID="cri-o://433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd" gracePeriod=30 Jul 22 19:12:20.044131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.044098 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccff4267-c5cd-4067-8de2-174870d1ed01" path="/var/lib/kubelet/pods/ccff4267-c5cd-4067-8de2-174870d1ed01/volumes" Jul 22 19:12:20.837747 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.837712 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688"] Jul 22 19:12:20.838284 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.838264 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccff4267-c5cd-4067-8de2-174870d1ed01" containerName="revoke-keys" Jul 22 19:12:20.838374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.838288 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccff4267-c5cd-4067-8de2-174870d1ed01" containerName="revoke-keys" Jul 22 19:12:20.838434 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.838414 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccff4267-c5cd-4067-8de2-174870d1ed01" containerName="revoke-keys" Jul 22 19:12:20.841673 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.841650 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.847292 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.847269 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-666c748b-gw-openshift-default-dockercfg-m66cw\"" Jul 22 19:12:20.860373 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.860343 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688"] Jul 22 19:12:20.970617 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970580 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-data\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.970617 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istiod-ca-cert\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.970903 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-envoy\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.970903 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970773 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-credential-socket\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.970903 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970804 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-token\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.970903 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970893 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-socket\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.971057 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-podinfo\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.971057 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xpsk\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-kube-api-access-6xpsk\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:20.971057 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:20.970999 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-certs\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.071928 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.071894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xpsk\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-kube-api-access-6xpsk\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.071928 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.071927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-certs\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072115 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-data\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072115 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istiod-ca-cert\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072183 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-envoy\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072183 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-credential-socket\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-token\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-socket\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-podinfo\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072304 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-certs\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072479 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072374 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-data\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072616 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072591 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-credential-socket\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072661 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072633 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-socket\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.072949 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.072922 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istiod-ca-cert\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.074336 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.074310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-envoy\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.074637 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.074619 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-podinfo\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.080192 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.080171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-token\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.080387 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.080371 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xpsk\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-kube-api-access-6xpsk\") pod \"e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.151660 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.151628 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:21.291385 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.291355 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688"] Jul 22 19:12:21.293241 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:12:21.293210 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59188e57_e08b_4b15_a4ac_18c1c1fb4be9.slice/crio-01f62a479077d0ee95468e2f1aa898bb8b5c558fa0cd21b77df369971994588d WatchSource:0}: Error finding container 01f62a479077d0ee95468e2f1aa898bb8b5c558fa0cd21b77df369971994588d: Status 404 returned error can't find the container with id 01f62a479077d0ee95468e2f1aa898bb8b5c558fa0cd21b77df369971994588d Jul 22 19:12:21.295413 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.295373 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:12:21.295526 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.295446 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:12:21.295526 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.295473 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:12:21.719559 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.719527 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" event={"ID":"59188e57-e08b-4b15-a4ac-18c1c1fb4be9","Type":"ContainerStarted","Data":"f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959"} Jul 22 19:12:21.719732 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.719578 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" event={"ID":"59188e57-e08b-4b15-a4ac-18c1c1fb4be9","Type":"ContainerStarted","Data":"01f62a479077d0ee95468e2f1aa898bb8b5c558fa0cd21b77df369971994588d"} Jul 22 19:12:21.737806 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:21.737762 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podStartSLOduration=1.737748414 podStartE2EDuration="1.737748414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:12:21.73629081 +0000 UTC m=+1950.254101737" watchObservedRunningTime="2026-07-22 19:12:21.737748414 +0000 UTC m=+1950.255559327" Jul 22 19:12:22.152109 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:22.152060 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:22.157245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:22.157219 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:22.725971 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:22.725936 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:22.726810 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:22.726792 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:24.995160 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:24.995096 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:12:25.113969 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.113914 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-credential-socket\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.113969 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.113972 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-data\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114364 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114002 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x5tq\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-kube-api-access-9x5tq\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114364 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114026 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a308f301-611e-4e73-9436-86ea4e354e6e-istio-podinfo\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114364 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114060 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-socket\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114364 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114091 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a308f301-611e-4e73-9436-86ea4e354e6e-istiod-ca-cert\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114364 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114120 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-certs\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114621 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114388 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-envoy\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114621 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114432 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-istio-token\") pod \"a308f301-611e-4e73-9436-86ea4e354e6e\" (UID: \"a308f301-611e-4e73-9436-86ea4e354e6e\") " Jul 22 19:12:25.114621 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114501 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:12:25.114788 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114620 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a308f301-611e-4e73-9436-86ea4e354e6e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:12:25.114924 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114508 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:12:25.114999 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.114945 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:12:25.115050 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.115019 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.115050 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.115038 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.115159 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.115125 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a308f301-611e-4e73-9436-86ea4e354e6e-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.115572 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.115543 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-data" (OuterVolumeSpecName: "istio-data") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:12:25.116554 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.116523 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-kube-api-access-9x5tq" (OuterVolumeSpecName: "kube-api-access-9x5tq") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "kube-api-access-9x5tq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:12:25.116653 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.116635 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:12:25.116695 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.116659 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:12:25.117069 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.117053 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a308f301-611e-4e73-9436-86ea4e354e6e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a308f301-611e-4e73-9436-86ea4e354e6e" (UID: "a308f301-611e-4e73-9436-86ea4e354e6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:12:25.215806 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.215769 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a308f301-611e-4e73-9436-86ea4e354e6e-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.215806 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.215804 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.216052 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.215844 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.216052 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.215859 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.216052 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.215870 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a308f301-611e-4e73-9436-86ea4e354e6e-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.216052 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.215882 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x5tq\" (UniqueName: \"kubernetes.io/projected/a308f301-611e-4e73-9436-86ea4e354e6e-kube-api-access-9x5tq\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:12:25.747479 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.747443 2566 generic.go:356] "Generic (PLEG): container finished" podID="a308f301-611e-4e73-9436-86ea4e354e6e" containerID="433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd" exitCode=0 Jul 22 19:12:25.747679 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.747494 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" Jul 22 19:12:25.747679 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.747536 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" event={"ID":"a308f301-611e-4e73-9436-86ea4e354e6e","Type":"ContainerDied","Data":"433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd"} Jul 22 19:12:25.747679 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.747600 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x" event={"ID":"a308f301-611e-4e73-9436-86ea4e354e6e","Type":"ContainerDied","Data":"6b65e10252d7ab60bf89399401c6c7d4bbef3cce7fc1f13ff2e9634634a622c2"} Jul 22 19:12:25.747679 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.747634 2566 scope.go:117] "RemoveContainer" containerID="433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd" Jul 22 19:12:25.757530 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.757514 2566 scope.go:117] "RemoveContainer" containerID="5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb" Jul 22 19:12:25.766014 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.765998 2566 scope.go:117] "RemoveContainer" containerID="433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd" Jul 22 19:12:25.766244 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:12:25.766228 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd\": container with ID starting with 433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd not found: ID does not exist" containerID="433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd" Jul 22 19:12:25.766288 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.766252 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd"} err="failed to get container status \"433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd\": rpc error: code = NotFound desc = could not find container \"433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd\": container with ID starting with 433be572cfa4498fac1bc922d3c6a1d5d32d34481465856fee44548703db40cd not found: ID does not exist" Jul 22 19:12:25.766288 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.766270 2566 scope.go:117] "RemoveContainer" containerID="5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb" Jul 22 19:12:25.766495 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:12:25.766481 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb\": container with ID starting with 5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb not found: ID does not exist" containerID="5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb" Jul 22 19:12:25.766607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.766499 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb"} err="failed to get container status \"5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb\": rpc error: code = NotFound desc = could not find container \"5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb\": container with ID starting with 5705ce962318aa99b6ac953b1117df896e9cf3b1bdcc88c009f4e586750113bb not found: ID does not exist" Jul 22 19:12:25.770676 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.770654 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x"] Jul 22 19:12:25.773835 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:25.773802 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-3c49b710-openshift-default-5df9887fcc-fxz4x"] Jul 22 19:12:26.044142 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:26.044075 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" path="/var/lib/kubelet/pods/a308f301-611e-4e73-9436-86ea4e354e6e/volumes" Jul 22 19:12:29.186037 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:29.186001 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:12:37.726773 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.726736 2566 patch_prober.go:28] interesting pod/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:12:37.727231 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.726807 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 19:12:37.817384 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.817356 2566 generic.go:356] "Generic (PLEG): container finished" podID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerID="f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959" exitCode=0 Jul 22 19:12:37.817546 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.817432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" event={"ID":"59188e57-e08b-4b15-a4ac-18c1c1fb4be9","Type":"ContainerDied","Data":"f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959"} Jul 22 19:12:37.817848 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.817832 2566 scope.go:117] "RemoveContainer" containerID="f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959" Jul 22 19:12:37.818795 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.818767 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:12:37.818884 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.818860 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:12:37.818923 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:37.818907 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:12:38.828959 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:38.828913 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" event={"ID":"59188e57-e08b-4b15-a4ac-18c1c1fb4be9","Type":"ContainerStarted","Data":"6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3"} Jul 22 19:12:39.152788 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:39.152753 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:39.153005 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:39.152981 2566 patch_prober.go:28] interesting pod/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:12:39.153054 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:39.153037 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 19:12:40.152177 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:40.152141 2566 patch_prober.go:28] interesting pod/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:12:40.152571 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:40.152194 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 19:12:41.152572 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:41.152537 2566 patch_prober.go:28] interesting pod/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:12:41.152969 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:41.152597 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 19:12:42.153041 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:42.153002 2566 patch_prober.go:28] interesting pod/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 19:12:42.153433 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:42.153066 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 19:12:43.156037 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:43.156007 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:43.156397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:43.156256 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:12:43.157007 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:12:43.156989 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:13:11.986047 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:11.986009 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:13:13.496196 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:13.496157 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688"] Jul 22 19:13:13.496679 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:13.496416 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" containerID="cri-o://6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3" gracePeriod=30 Jul 22 19:13:14.315475 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.315440 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2"] Jul 22 19:13:14.315858 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.315809 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" Jul 22 19:13:14.315858 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.315841 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" Jul 22 19:13:14.315962 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.315924 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" Jul 22 19:13:14.319106 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.319088 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.321287 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.321270 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9f921dcc-openshift-default-dockercfg-2h6bl\"" Jul 22 19:13:14.332049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.332024 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2"] Jul 22 19:13:14.457103 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-data\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457103 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457106 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d4dt\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-kube-api-access-6d4dt\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457127 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-token\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-certs\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457243 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istiod-ca-cert\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457296 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-credential-socket\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-envoy\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457597 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-podinfo\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.457597 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.457411 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-socket\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.565024 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.564990 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-data\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.565603 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.565551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d4dt\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-kube-api-access-6d4dt\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.565603 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.565598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-token\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.565870 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.565844 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-certs\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.565945 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.565861 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-data\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.565945 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.565926 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istiod-ca-cert\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566032 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.565967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-credential-socket\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566032 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566010 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-envoy\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566134 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-podinfo\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566134 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-certs\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566242 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-socket\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566302 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566282 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-credential-socket\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566531 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566508 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-socket\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.566659 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.566630 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istiod-ca-cert\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.568411 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.568392 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-podinfo\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.568519 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.568472 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-envoy\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.581256 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.581233 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-token\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.581428 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.581406 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d4dt\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-kube-api-access-6d4dt\") pod \"e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.630569 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.630550 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:14.768192 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.768164 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2"] Jul 22 19:13:14.770052 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:13:14.770020 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9dde01_60f7_4eaf_9ecd_957208e1a522.slice/crio-26f39d24b8dbd678ad8998ebaebd365d5e0808d4648b7ae6d8ff160934c4804a WatchSource:0}: Error finding container 26f39d24b8dbd678ad8998ebaebd365d5e0808d4648b7ae6d8ff160934c4804a: Status 404 returned error can't find the container with id 26f39d24b8dbd678ad8998ebaebd365d5e0808d4648b7ae6d8ff160934c4804a Jul 22 19:13:14.772063 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.772033 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:13:14.772120 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.772098 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:13:14.772158 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:14.772128 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:13:15.035003 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:15.034955 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" event={"ID":"1e9dde01-60f7-4eaf-9ecd-957208e1a522","Type":"ContainerStarted","Data":"f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6"} Jul 22 19:13:15.035184 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:15.035021 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" event={"ID":"1e9dde01-60f7-4eaf-9ecd-957208e1a522","Type":"ContainerStarted","Data":"26f39d24b8dbd678ad8998ebaebd365d5e0808d4648b7ae6d8ff160934c4804a"} Jul 22 19:13:15.057103 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:15.057055 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podStartSLOduration=1.057039381 podStartE2EDuration="1.057039381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:13:15.05472419 +0000 UTC m=+2003.572535098" watchObservedRunningTime="2026-07-22 19:13:15.057039381 +0000 UTC m=+2003.574850294" Jul 22 19:13:15.631432 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:15.631398 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:15.635901 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:15.635877 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:16.043740 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:16.043663 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:16.043740 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:16.043737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:18.754173 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.754150 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:13:18.907703 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907675 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-certs\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.907922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907727 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-socket\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.907922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907750 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-podinfo\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.907922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907788 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-data\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.907922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907838 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-credential-socket\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.907922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907864 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istiod-ca-cert\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.907922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907905 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-envoy\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.908252 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907928 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xpsk\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-kube-api-access-6xpsk\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.908252 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.907957 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-token\") pod \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\" (UID: \"59188e57-e08b-4b15-a4ac-18c1c1fb4be9\") " Jul 22 19:13:18.908252 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908041 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:13:18.908252 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908086 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:13:18.908252 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908165 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:13:18.908511 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908315 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:18.908511 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908334 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:18.908511 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908349 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:18.908511 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908381 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:13:18.908849 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.908799 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-data" (OuterVolumeSpecName: "istio-data") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:13:18.910180 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.910155 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-kube-api-access-6xpsk" (OuterVolumeSpecName: "kube-api-access-6xpsk") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "kube-api-access-6xpsk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:13:18.910448 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.910426 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:13:18.910568 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.910496 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-token" (OuterVolumeSpecName: "istio-token") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:13:18.910568 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:18.910499 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59188e57-e08b-4b15-a4ac-18c1c1fb4be9" (UID: "59188e57-e08b-4b15-a4ac-18c1c1fb4be9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:13:19.009099 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.009063 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:19.009099 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.009091 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:19.009099 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.009101 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:19.009099 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.009109 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:19.009347 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.009120 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xpsk\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-kube-api-access-6xpsk\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:19.009347 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.009129 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59188e57-e08b-4b15-a4ac-18c1c1fb4be9-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:13:19.059441 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.059408 2566 generic.go:356] "Generic (PLEG): container finished" podID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerID="6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3" exitCode=0 Jul 22 19:13:19.059592 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.059470 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" Jul 22 19:13:19.059592 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.059495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" event={"ID":"59188e57-e08b-4b15-a4ac-18c1c1fb4be9","Type":"ContainerDied","Data":"6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3"} Jul 22 19:13:19.059592 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.059557 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688" event={"ID":"59188e57-e08b-4b15-a4ac-18c1c1fb4be9","Type":"ContainerDied","Data":"01f62a479077d0ee95468e2f1aa898bb8b5c558fa0cd21b77df369971994588d"} Jul 22 19:13:19.059592 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.059588 2566 scope.go:117] "RemoveContainer" containerID="6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3" Jul 22 19:13:19.069417 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.069393 2566 scope.go:117] "RemoveContainer" containerID="f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959" Jul 22 19:13:19.077780 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.077762 2566 scope.go:117] "RemoveContainer" containerID="6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3" Jul 22 19:13:19.078057 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:13:19.078033 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3\": container with ID starting with 6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3 not found: ID does not exist" containerID="6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3" Jul 22 19:13:19.078156 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.078060 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3"} err="failed to get container status \"6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3\": rpc error: code = NotFound desc = could not find container \"6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3\": container with ID starting with 6f93de9262976f1a5b52394515f0a3d32082b90ae1507a353ddaadb1ca44f4e3 not found: ID does not exist" Jul 22 19:13:19.078156 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.078080 2566 scope.go:117] "RemoveContainer" containerID="f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959" Jul 22 19:13:19.078915 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:13:19.078887 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959\": container with ID starting with f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959 not found: ID does not exist" containerID="f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959" Jul 22 19:13:19.078989 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.078922 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959"} err="failed to get container status \"f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959\": rpc error: code = NotFound desc = could not find container \"f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959\": container with ID starting with f510d4905a292dd16f3be4b24cf0f892fcc5132e6961aca842246273fb0dc959 not found: ID does not exist" Jul 22 19:13:19.081581 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.081560 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688"] Jul 22 19:13:19.085495 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.085472 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-666c748b-gw-openshift-default-6db5d748bc-jl688"] Jul 22 19:13:19.989460 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:19.989422 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:13:20.044232 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:20.044194 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" path="/var/lib/kubelet/pods/59188e57-e08b-4b15-a4ac-18c1c1fb4be9/volumes" Jul 22 19:13:28.109910 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:28.109801 2566 generic.go:356] "Generic (PLEG): container finished" podID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerID="f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6" exitCode=0 Jul 22 19:13:28.109910 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:28.109877 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" event={"ID":"1e9dde01-60f7-4eaf-9ecd-957208e1a522","Type":"ContainerDied","Data":"f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6"} Jul 22 19:13:28.110329 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:28.110204 2566 scope.go:117] "RemoveContainer" containerID="f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6" Jul 22 19:13:28.111162 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:28.111131 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:13:28.111232 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:28.111197 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:13:28.111274 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:28.111228 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:13:29.120139 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:29.120099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" event={"ID":"1e9dde01-60f7-4eaf-9ecd-957208e1a522","Type":"ContainerStarted","Data":"2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0"} Jul 22 19:13:29.632536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:29.632503 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:29.632946 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:29.632919 2566 patch_prober.go:28] interesting pod/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:13:29.633148 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:29.633121 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 19:13:30.631284 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:30.631239 2566 patch_prober.go:28] interesting pod/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:13:30.631806 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:30.631313 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 19:13:31.041400 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:31.041316 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:31.631364 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:31.631320 2566 patch_prober.go:28] interesting pod/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:13:31.631877 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:31.631409 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 19:13:32.634227 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:32.634188 2566 patch_prober.go:28] interesting pod/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 22 19:13:32.634734 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:32.634256 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 22 19:13:33.635167 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:33.635137 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:33.636139 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:33.636119 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:13:55.985035 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:55.984996 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:13:58.703307 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:58.703274 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2"] Jul 22 19:13:58.703736 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:58.703504 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" containerID="cri-o://2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0" gracePeriod=30 Jul 22 19:13:59.554002 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.553971 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj"] Jul 22 19:13:59.554378 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554366 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" Jul 22 19:13:59.554423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554381 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" Jul 22 19:13:59.554423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554389 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" Jul 22 19:13:59.554423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554395 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" Jul 22 19:13:59.554423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554418 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" Jul 22 19:13:59.554550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554426 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" Jul 22 19:13:59.554550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554489 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a308f301-611e-4e73-9436-86ea4e354e6e" containerName="istio-proxy" Jul 22 19:13:59.554550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.554500 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" Jul 22 19:13:59.557512 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.557491 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.560130 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.560109 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-908afcb9-openshift-default-dockercfg-2sk99\"" Jul 22 19:13:59.569924 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.569900 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj"] Jul 22 19:13:59.691161 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41580b4c-163b-42ee-9142-d2ce81a7f87e-istiod-ca-cert\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691351 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691220 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-token\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691351 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-socket\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691351 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-data\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691351 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691323 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wn2n\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-kube-api-access-6wn2n\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691555 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-podinfo\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691555 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691404 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-credential-socket\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691555 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691446 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-certs\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.691555 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.691493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-envoy\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792513 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792483 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-envoy\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792522 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41580b4c-163b-42ee-9142-d2ce81a7f87e-istiod-ca-cert\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-token\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792607 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-socket\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792625 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-data\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wn2n\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-kube-api-access-6wn2n\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-podinfo\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-credential-socket\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.792986 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.792718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-certs\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.793456 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.793037 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-socket\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.793456 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.793162 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-data\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.793456 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.793203 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-certs\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.793456 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.793347 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-credential-socket\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.793456 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.793444 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41580b4c-163b-42ee-9142-d2ce81a7f87e-istiod-ca-cert\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.794854 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.794813 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-envoy\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.795162 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.795146 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-podinfo\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.800648 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.800620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wn2n\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-kube-api-access-6wn2n\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.800737 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.800647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-token\") pod \"e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:13:59.867966 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:13:59.867937 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:00.005414 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.005390 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj"] Jul 22 19:14:00.007710 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:14:00.007678 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41580b4c_163b_42ee_9142_d2ce81a7f87e.slice/crio-de90f61018404d06c01fd3d2ff342d620be941741d99c90716a820015b20ede1 WatchSource:0}: Error finding container de90f61018404d06c01fd3d2ff342d620be941741d99c90716a820015b20ede1: Status 404 returned error can't find the container with id de90f61018404d06c01fd3d2ff342d620be941741d99c90716a820015b20ede1 Jul 22 19:14:00.009831 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.009781 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:14:00.009936 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.009886 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:14:00.010003 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.009932 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:14:00.298120 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.298030 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" event={"ID":"41580b4c-163b-42ee-9142-d2ce81a7f87e","Type":"ContainerStarted","Data":"c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5"} Jul 22 19:14:00.298120 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.298092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" event={"ID":"41580b4c-163b-42ee-9142-d2ce81a7f87e","Type":"ContainerStarted","Data":"de90f61018404d06c01fd3d2ff342d620be941741d99c90716a820015b20ede1"} Jul 22 19:14:00.318219 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.318174 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" podStartSLOduration=1.318160213 podStartE2EDuration="1.318160213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:14:00.315034835 +0000 UTC m=+2048.832845743" watchObservedRunningTime="2026-07-22 19:14:00.318160213 +0000 UTC m=+2048.835971179" Jul 22 19:14:00.868896 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:00.868865 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:01.877231 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:01.877196 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:02.310214 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:02.310130 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:02.311229 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:02.311204 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:03.638072 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:03.638032 2566 patch_prober.go:28] interesting pod/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:14:03.638442 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:03.638086 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:14:03.949437 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:03.949415 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:14:04.031545 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031512 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-socket\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031545 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031552 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d4dt\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-kube-api-access-6d4dt\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031575 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-envoy\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031608 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-credential-socket\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031653 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-podinfo\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031691 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-data\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031740 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-token\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.031781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031761 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-certs\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.032100 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031792 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istiod-ca-cert\") pod \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\" (UID: \"1e9dde01-60f7-4eaf-9ecd-957208e1a522\") " Jul 22 19:14:04.032100 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.031795 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:04.032100 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032004 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:04.032100 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032027 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:04.032298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032217 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.032298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032240 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.032298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032245 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:14:04.032298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032256 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.032684 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.032653 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:04.033950 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.033919 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-kube-api-access-6d4dt" (OuterVolumeSpecName: "kube-api-access-6d4dt") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "kube-api-access-6d4dt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:14:04.034284 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.034259 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:14:04.034397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.034350 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:04.034397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.034350 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e9dde01-60f7-4eaf-9ecd-957208e1a522" (UID: "1e9dde01-60f7-4eaf-9ecd-957208e1a522"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:14:04.133374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.133333 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.133374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.133365 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.133374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.133377 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.133596 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.133385 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.133596 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.133395 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d4dt\" (UniqueName: \"kubernetes.io/projected/1e9dde01-60f7-4eaf-9ecd-957208e1a522-kube-api-access-6d4dt\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.133596 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.133403 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e9dde01-60f7-4eaf-9ecd-957208e1a522-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:04.322029 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.321928 2566 generic.go:356] "Generic (PLEG): container finished" podID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerID="2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0" exitCode=0 Jul 22 19:14:04.322029 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.321989 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" Jul 22 19:14:04.322029 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.322014 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" event={"ID":"1e9dde01-60f7-4eaf-9ecd-957208e1a522","Type":"ContainerDied","Data":"2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0"} Jul 22 19:14:04.322314 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.322068 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2" event={"ID":"1e9dde01-60f7-4eaf-9ecd-957208e1a522","Type":"ContainerDied","Data":"26f39d24b8dbd678ad8998ebaebd365d5e0808d4648b7ae6d8ff160934c4804a"} Jul 22 19:14:04.322314 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.322090 2566 scope.go:117] "RemoveContainer" containerID="2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0" Jul 22 19:14:04.331654 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.331636 2566 scope.go:117] "RemoveContainer" containerID="f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6" Jul 22 19:14:04.339482 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.339458 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2"] Jul 22 19:14:04.341088 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.340980 2566 scope.go:117] "RemoveContainer" containerID="2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0" Jul 22 19:14:04.341534 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:14:04.341509 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0\": container with ID starting with 2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0 not found: ID does not exist" containerID="2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0" Jul 22 19:14:04.341619 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.341542 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0"} err="failed to get container status \"2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0\": rpc error: code = NotFound desc = could not find container \"2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0\": container with ID starting with 2e7ea59505646de53510c990d242c601a34653c452cd5d91f2a6906a0a4b35c0 not found: ID does not exist" Jul 22 19:14:04.341619 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.341560 2566 scope.go:117] "RemoveContainer" containerID="f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6" Jul 22 19:14:04.341810 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:14:04.341786 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6\": container with ID starting with f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6 not found: ID does not exist" containerID="f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6" Jul 22 19:14:04.341890 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.341836 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6"} err="failed to get container status \"f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6\": rpc error: code = NotFound desc = could not find container \"f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6\": container with ID starting with f694419658a4eebe693d0193c0db206e25a27efdffced8b17333c88389ef23a6 not found: ID does not exist" Jul 22 19:14:04.343122 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:04.343104 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9f921dcc-openshift-default-77fc847465-mqms2"] Jul 22 19:14:06.049713 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:06.049676 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" path="/var/lib/kubelet/pods/1e9dde01-60f7-4eaf-9ecd-957208e1a522/volumes" Jul 22 19:14:07.028984 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:07.028947 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:14:15.391529 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:15.391496 2566 generic.go:356] "Generic (PLEG): container finished" podID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerID="c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5" exitCode=0 Jul 22 19:14:15.392021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:15.391576 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" event={"ID":"41580b4c-163b-42ee-9142-d2ce81a7f87e","Type":"ContainerDied","Data":"c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5"} Jul 22 19:14:15.392021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:15.391993 2566 scope.go:117] "RemoveContainer" containerID="c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5" Jul 22 19:14:15.392987 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:15.392959 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:14:15.393055 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:15.393028 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:14:15.393092 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:15.393057 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:14:16.399020 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:16.398986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" event={"ID":"41580b4c-163b-42ee-9142-d2ce81a7f87e","Type":"ContainerStarted","Data":"43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e"} Jul 22 19:14:16.868304 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:16.868275 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:16.869501 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:16.869475 2566 patch_prober.go:28] interesting pod/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:14:16.869636 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:16.869533 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 19:14:17.311173 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:17.311071 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:17.868678 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:17.868638 2566 patch_prober.go:28] interesting pod/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:14:17.869184 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:17.868700 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 19:14:18.868635 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:18.868601 2566 patch_prober.go:28] interesting pod/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:14:18.868813 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:18.868657 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 19:14:19.868758 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:19.868722 2566 patch_prober.go:28] interesting pod/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 19:14:19.868959 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:19.868780 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 19:14:20.872595 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:20.872554 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:20.873507 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:20.873487 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:39.993425 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:39.993389 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:14:42.129637 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:42.129601 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj"] Jul 22 19:14:42.130065 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:42.129860 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" containerID="cri-o://43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e" gracePeriod=30 Jul 22 19:14:47.379149 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.379117 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:47.530536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530456 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-socket\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530496 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-token\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530535 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-podinfo\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530793 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41580b4c-163b-42ee-9142-d2ce81a7f87e-istiod-ca-cert\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530793 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530640 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wn2n\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-kube-api-access-6wn2n\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530793 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530671 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-certs\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530793 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530706 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-data\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.530793 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530736 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-credential-socket\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.531131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530791 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:47.531131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.530797 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-envoy\") pod \"41580b4c-163b-42ee-9142-d2ce81a7f87e\" (UID: \"41580b4c-163b-42ee-9142-d2ce81a7f87e\") " Jul 22 19:14:47.531131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531095 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:47.531271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531123 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41580b4c-163b-42ee-9142-d2ce81a7f87e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:14:47.531330 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531285 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.531330 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531305 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41580b4c-163b-42ee-9142-d2ce81a7f87e-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.531330 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531321 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.531547 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531525 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:47.531639 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.531532 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-data" (OuterVolumeSpecName: "istio-data") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:47.533229 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.533209 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-token" (OuterVolumeSpecName: "istio-token") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:14:47.533363 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.533347 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:14:47.533425 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.533365 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:14:47.533482 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.533462 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-kube-api-access-6wn2n" (OuterVolumeSpecName: "kube-api-access-6wn2n") pod "41580b4c-163b-42ee-9142-d2ce81a7f87e" (UID: "41580b4c-163b-42ee-9142-d2ce81a7f87e"). InnerVolumeSpecName "kube-api-access-6wn2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:14:47.579307 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.579272 2566 generic.go:356] "Generic (PLEG): container finished" podID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerID="43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e" exitCode=0 Jul 22 19:14:47.579488 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.579327 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" Jul 22 19:14:47.579488 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.579353 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" event={"ID":"41580b4c-163b-42ee-9142-d2ce81a7f87e","Type":"ContainerDied","Data":"43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e"} Jul 22 19:14:47.579488 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.579410 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj" event={"ID":"41580b4c-163b-42ee-9142-d2ce81a7f87e","Type":"ContainerDied","Data":"de90f61018404d06c01fd3d2ff342d620be941741d99c90716a820015b20ede1"} Jul 22 19:14:47.579488 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.579432 2566 scope.go:117] "RemoveContainer" containerID="43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e" Jul 22 19:14:47.589389 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.589371 2566 scope.go:117] "RemoveContainer" containerID="c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5" Jul 22 19:14:47.599809 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.599791 2566 scope.go:117] "RemoveContainer" containerID="43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e" Jul 22 19:14:47.600100 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:14:47.600082 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e\": container with ID starting with 43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e not found: ID does not exist" containerID="43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e" Jul 22 19:14:47.600193 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.600113 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e"} err="failed to get container status \"43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e\": rpc error: code = NotFound desc = could not find container \"43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e\": container with ID starting with 43ed850eaa7e7f9ae23517ed19fbe6a9f00abf506c8c204e727481f7a34d3c0e not found: ID does not exist" Jul 22 19:14:47.600193 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.600138 2566 scope.go:117] "RemoveContainer" containerID="c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5" Jul 22 19:14:47.600429 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:14:47.600406 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5\": container with ID starting with c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5 not found: ID does not exist" containerID="c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5" Jul 22 19:14:47.600505 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.600438 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5"} err="failed to get container status \"c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5\": rpc error: code = NotFound desc = could not find container \"c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5\": container with ID starting with c97405b2e50fb9796bcf0ce146fbc6b0d035780b16b7f72b59e255c37c05d0a5 not found: ID does not exist" Jul 22 19:14:47.601696 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.601671 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj"] Jul 22 19:14:47.604778 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.604755 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-908afcb9-openshift-default-555fc8ff47-tt4cj"] Jul 22 19:14:47.632720 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.632694 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.632720 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.632718 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.632926 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.632731 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.632926 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.632744 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wn2n\" (UniqueName: \"kubernetes.io/projected/41580b4c-163b-42ee-9142-d2ce81a7f87e-kube-api-access-6wn2n\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.632926 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.632760 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:47.632926 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:47.632772 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41580b4c-163b-42ee-9142-d2ce81a7f87e-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:14:48.044256 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:48.044225 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" path="/var/lib/kubelet/pods/41580b4c-163b-42ee-9142-d2ce81a7f87e/volumes" Jul 22 19:14:52.157037 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:52.157009 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:14:52.163111 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:52.163089 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:14:52.173200 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:52.173182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:14:52.179442 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:14:52.179427 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:16:09.511185 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:09.511147 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:16:19.385345 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:19.385304 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:16:21.036230 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036198 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf"] Jul 22 19:16:21.036706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036603 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" Jul 22 19:16:21.036706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036622 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" Jul 22 19:16:21.036706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036646 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" Jul 22 19:16:21.036706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036656 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" Jul 22 19:16:21.036706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036665 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" Jul 22 19:16:21.036706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036673 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" Jul 22 19:16:21.037054 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036766 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" Jul 22 19:16:21.037054 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036775 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" Jul 22 19:16:21.037054 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036788 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e9dde01-60f7-4eaf-9ecd-957208e1a522" containerName="istio-proxy" Jul 22 19:16:21.037054 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.036795 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="59188e57-e08b-4b15-a4ac-18c1c1fb4be9" containerName="istio-proxy" Jul 22 19:16:21.040234 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.040209 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.042355 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.042341 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3579e215-openshift-default-dockercfg-qzp78\"" Jul 22 19:16:21.052523 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.052496 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf"] Jul 22 19:16:21.084941 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.084912 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-envoy\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085053 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.084960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkkhm\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-kube-api-access-wkkhm\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085095 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085063 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-certs\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085098 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-socket\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-credential-socket\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085223 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085130 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2faeb50a-5681-4a79-901e-8d7a7b57df39-istiod-ca-cert\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085278 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-data\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085345 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-token\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.085490 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.085380 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-podinfo\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.185932 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.185901 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-token\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186109 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.185942 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-podinfo\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186109 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186022 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-envoy\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186109 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186065 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkkhm\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-kube-api-access-wkkhm\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186109 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186105 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-certs\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186329 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-socket\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186329 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186262 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-credential-socket\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186329 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2faeb50a-5681-4a79-901e-8d7a7b57df39-istiod-ca-cert\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186476 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186358 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-data\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186585 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-socket\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186712 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186638 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-certs\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186793 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186721 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-data\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.186877 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.186848 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-credential-socket\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.187182 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.187149 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2faeb50a-5681-4a79-901e-8d7a7b57df39-istiod-ca-cert\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.188431 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.188412 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-envoy\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.188624 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.188599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-podinfo\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.194075 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.194039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-token\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.194382 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.194362 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkkhm\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-kube-api-access-wkkhm\") pod \"e2e-mt-3579e215-openshift-default-b4449f887-bmpwf\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.352250 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.352163 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:21.500943 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.500916 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf"] Jul 22 19:16:21.502307 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:16:21.502269 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2faeb50a_5681_4a79_901e_8d7a7b57df39.slice/crio-6017b824b99d8e07f2ca10aee2b0c6e88281da1cd5899339c6c35047624caaff WatchSource:0}: Error finding container 6017b824b99d8e07f2ca10aee2b0c6e88281da1cd5899339c6c35047624caaff: Status 404 returned error can't find the container with id 6017b824b99d8e07f2ca10aee2b0c6e88281da1cd5899339c6c35047624caaff Jul 22 19:16:21.504523 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.504486 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:16:21.504604 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.504556 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:16:21.504604 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:21.504600 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:16:22.089916 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:22.089875 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" event={"ID":"2faeb50a-5681-4a79-901e-8d7a7b57df39","Type":"ContainerStarted","Data":"0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b"} Jul 22 19:16:22.090317 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:22.089933 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" event={"ID":"2faeb50a-5681-4a79-901e-8d7a7b57df39","Type":"ContainerStarted","Data":"6017b824b99d8e07f2ca10aee2b0c6e88281da1cd5899339c6c35047624caaff"} Jul 22 19:16:22.110055 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:22.109995 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podStartSLOduration=1.109978678 podStartE2EDuration="1.109978678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:16:22.107225236 +0000 UTC m=+2190.625036150" watchObservedRunningTime="2026-07-22 19:16:22.109978678 +0000 UTC m=+2190.627789592" Jul 22 19:16:22.352772 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:22.352668 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:22.357682 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:22.357648 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:23.095854 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:23.095806 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:23.096915 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:23.096897 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:30.497740 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:30.497700 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:16:38.098053 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.098016 2566 patch_prober.go:28] interesting pod/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 19:16:38.098425 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.098081 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 19:16:38.180021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.179978 2566 generic.go:356] "Generic (PLEG): container finished" podID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerID="0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b" exitCode=0 Jul 22 19:16:38.180202 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.180057 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" event={"ID":"2faeb50a-5681-4a79-901e-8d7a7b57df39","Type":"ContainerDied","Data":"0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b"} Jul 22 19:16:38.180497 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.180480 2566 scope.go:117] "RemoveContainer" containerID="0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b" Jul 22 19:16:38.184389 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.181746 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:16:38.184389 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.181859 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:16:38.184389 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:38.181924 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:16:39.187056 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:39.187011 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" event={"ID":"2faeb50a-5681-4a79-901e-8d7a7b57df39","Type":"ContainerStarted","Data":"a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272"} Jul 22 19:16:39.353245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:39.353204 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:39.353431 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:39.353402 2566 patch_prober.go:28] interesting pod/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 19:16:39.353476 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:39.353451 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 19:16:40.353019 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:40.352982 2566 patch_prober.go:28] interesting pod/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 19:16:40.353402 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:40.353068 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 19:16:41.353154 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:41.353121 2566 patch_prober.go:28] interesting pod/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 19:16:41.353547 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:41.353176 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 19:16:42.352772 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:42.352739 2566 patch_prober.go:28] interesting pod/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 19:16:42.352979 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:42.352791 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 19:16:43.357407 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:43.357377 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:43.357869 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:43.357629 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:43.358432 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:43.358415 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:16:49.518275 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:49.518237 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:16:57.534556 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:57.534518 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf"] Jul 22 19:16:57.535116 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:57.534758 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" containerID="cri-o://a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272" gracePeriod=30 Jul 22 19:16:58.359856 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:58.359794 2566 patch_prober.go:28] interesting pod/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:16:58.360060 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:16:58.359873 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:17:02.791991 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.791967 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:17:02.972709 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.972677 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2faeb50a-5681-4a79-901e-8d7a7b57df39-istiod-ca-cert\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.972945 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.972725 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkkhm\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-kube-api-access-wkkhm\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.972945 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.972901 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-credential-socket\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973036 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.972960 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-socket\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973036 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973018 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-podinfo\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973036 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973022 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2faeb50a-5681-4a79-901e-8d7a7b57df39-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:17:02.973202 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973057 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-data\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973202 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973108 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-certs\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973202 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973152 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-envoy\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973202 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973165 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:17:02.973403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973177 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-token\") pod \"2faeb50a-5681-4a79-901e-8d7a7b57df39\" (UID: \"2faeb50a-5681-4a79-901e-8d7a7b57df39\") " Jul 22 19:17:02.973403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973255 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:17:02.973569 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973540 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:02.973651 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973577 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:02.973651 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973596 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2faeb50a-5681-4a79-901e-8d7a7b57df39-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:02.973651 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973609 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:17:02.974016 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.973992 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-data" (OuterVolumeSpecName: "istio-data") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:17:02.975387 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.975361 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:17:02.975387 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.975379 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-token" (OuterVolumeSpecName: "istio-token") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:17:02.975505 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.975464 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:17:02.975505 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:02.975468 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-kube-api-access-wkkhm" (OuterVolumeSpecName: "kube-api-access-wkkhm") pod "2faeb50a-5681-4a79-901e-8d7a7b57df39" (UID: "2faeb50a-5681-4a79-901e-8d7a7b57df39"). InnerVolumeSpecName "kube-api-access-wkkhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:17:03.074366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.074332 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:03.074366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.074357 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:03.074366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.074367 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:03.074366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.074377 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:03.074638 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.074385 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:03.074638 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.074394 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkkhm\" (UniqueName: \"kubernetes.io/projected/2faeb50a-5681-4a79-901e-8d7a7b57df39-kube-api-access-wkkhm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:17:03.318060 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.317976 2566 generic.go:356] "Generic (PLEG): container finished" podID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerID="a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272" exitCode=0 Jul 22 19:17:03.318060 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.318018 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" event={"ID":"2faeb50a-5681-4a79-901e-8d7a7b57df39","Type":"ContainerDied","Data":"a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272"} Jul 22 19:17:03.318060 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.318029 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" Jul 22 19:17:03.318282 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.318074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf" event={"ID":"2faeb50a-5681-4a79-901e-8d7a7b57df39","Type":"ContainerDied","Data":"6017b824b99d8e07f2ca10aee2b0c6e88281da1cd5899339c6c35047624caaff"} Jul 22 19:17:03.318282 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.318107 2566 scope.go:117] "RemoveContainer" containerID="a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272" Jul 22 19:17:03.327767 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.327751 2566 scope.go:117] "RemoveContainer" containerID="0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b" Jul 22 19:17:03.335733 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.335715 2566 scope.go:117] "RemoveContainer" containerID="a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272" Jul 22 19:17:03.336029 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:17:03.336010 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272\": container with ID starting with a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272 not found: ID does not exist" containerID="a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272" Jul 22 19:17:03.336090 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.336040 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272"} err="failed to get container status \"a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272\": rpc error: code = NotFound desc = could not find container \"a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272\": container with ID starting with a711de650318d9a7cadaeea2c1e6170d0fc151560d53515c5c9bed51880e7272 not found: ID does not exist" Jul 22 19:17:03.336090 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.336062 2566 scope.go:117] "RemoveContainer" containerID="0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b" Jul 22 19:17:03.336336 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:17:03.336305 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b\": container with ID starting with 0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b not found: ID does not exist" containerID="0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b" Jul 22 19:17:03.336426 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.336340 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b"} err="failed to get container status \"0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b\": rpc error: code = NotFound desc = could not find container \"0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b\": container with ID starting with 0b3a47cf5293d9c87985043769ce28d4d3f4d11573f15192f319dfcaa0e05e6b not found: ID does not exist" Jul 22 19:17:03.339119 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.339096 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf"] Jul 22 19:17:03.343439 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:03.343414 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3579e215-openshift-default-b4449f887-bmpwf"] Jul 22 19:17:04.043957 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:04.043920 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" path="/var/lib/kubelet/pods/2faeb50a-5681-4a79-901e-8d7a7b57df39/volumes" Jul 22 19:17:17.574473 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.574393 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b"] Jul 22 19:17:17.575025 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.574966 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" Jul 22 19:17:17.575025 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.574990 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" Jul 22 19:17:17.575148 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.575026 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" Jul 22 19:17:17.575148 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.575036 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" Jul 22 19:17:17.575148 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.575127 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" Jul 22 19:17:17.575148 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.575141 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" Jul 22 19:17:17.575343 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.575153 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="41580b4c-163b-42ee-9142-d2ce81a7f87e" containerName="istio-proxy" Jul 22 19:17:17.579725 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.579703 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.581991 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.581972 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-73899c00-openshift-default-dockercfg-n5fc9\"" Jul 22 19:17:17.590293 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.590258 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b"] Jul 22 19:17:17.694605 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694576 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm5wg\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-kube-api-access-bm5wg\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694786 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istiod-ca-cert\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694786 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694658 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-podinfo\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694786 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694727 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-certs\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694786 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694759 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-data\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694979 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694781 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-credential-socket\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694979 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-envoy\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694979 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-token\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.694979 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.694898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-socket\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795588 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-podinfo\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795745 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795596 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-certs\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795745 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795616 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-data\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795745 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-credential-socket\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795745 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-envoy\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795745 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-token\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.795745 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-socket\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796097 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795781 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm5wg\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-kube-api-access-bm5wg\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796097 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.795861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istiod-ca-cert\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796097 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.796075 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-certs\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796249 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.796193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-data\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796313 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.796252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-credential-socket\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796313 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.796287 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-socket\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.796536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.796519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istiod-ca-cert\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.798253 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.798227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-envoy\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.798476 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.798458 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-podinfo\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.803457 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.803432 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-token\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.803584 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.803567 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm5wg\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-kube-api-access-bm5wg\") pod \"e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:17.892416 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:17.892387 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:18.029787 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.029759 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b"] Jul 22 19:17:18.031850 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:17:18.031801 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc10e56_e82b_4a4c_9261_0f2671e9673e.slice/crio-1d4f0f9a291d3bda94fe9f70b5d334215a0bc76640bfeb04a70dce18873aa620 WatchSource:0}: Error finding container 1d4f0f9a291d3bda94fe9f70b5d334215a0bc76640bfeb04a70dce18873aa620: Status 404 returned error can't find the container with id 1d4f0f9a291d3bda94fe9f70b5d334215a0bc76640bfeb04a70dce18873aa620 Jul 22 19:17:18.034174 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.034139 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:17:18.034259 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.034204 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:17:18.034259 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.034231 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:17:18.401132 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.401092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" event={"ID":"1fc10e56-e82b-4a4c-9261-0f2671e9673e","Type":"ContainerStarted","Data":"18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7"} Jul 22 19:17:18.401297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.401155 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" event={"ID":"1fc10e56-e82b-4a4c-9261-0f2671e9673e","Type":"ContainerStarted","Data":"1d4f0f9a291d3bda94fe9f70b5d334215a0bc76640bfeb04a70dce18873aa620"} Jul 22 19:17:18.421431 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.421375 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podStartSLOduration=1.421358863 podStartE2EDuration="1.421358863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:17:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:17:18.4192323 +0000 UTC m=+2246.937043215" watchObservedRunningTime="2026-07-22 19:17:18.421358863 +0000 UTC m=+2246.939169776" Jul 22 19:17:18.892825 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:18.892789 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:19.898027 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:19.897995 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:20.412562 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:20.412524 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:20.413564 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:20.413547 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:26.686092 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:26.686060 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:17:34.489297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:34.489220 2566 generic.go:356] "Generic (PLEG): container finished" podID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerID="18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7" exitCode=0 Jul 22 19:17:34.489680 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:34.489295 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" event={"ID":"1fc10e56-e82b-4a4c-9261-0f2671e9673e","Type":"ContainerDied","Data":"18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7"} Jul 22 19:17:34.489749 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:34.489734 2566 scope.go:117] "RemoveContainer" containerID="18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7" Jul 22 19:17:34.490781 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:34.490749 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:17:34.490861 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:34.490842 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:17:34.490905 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:34.490884 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:17:35.413127 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:35.413086 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:35.488729 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:35.488692 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:17:35.496807 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:35.496772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" event={"ID":"1fc10e56-e82b-4a4c-9261-0f2671e9673e","Type":"ContainerStarted","Data":"ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64"} Jul 22 19:17:35.893523 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:35.893495 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:35.894576 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:35.894550 2566 patch_prober.go:28] interesting pod/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:17:35.894704 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:35.894612 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 19:17:36.893570 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:36.893531 2566 patch_prober.go:28] interesting pod/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:17:36.893982 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:36.893595 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 19:17:37.893216 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:37.893175 2566 patch_prober.go:28] interesting pod/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:17:37.893392 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:37.893249 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 19:17:38.893752 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:38.893713 2566 patch_prober.go:28] interesting pod/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 19:17:38.894172 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:38.893775 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 19:17:39.897221 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:39.897187 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:39.897634 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:39.897485 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:17:39.898289 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:17:39.898266 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:18:00.313840 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:18:00.313796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:18:27.486019 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:18:27.485982 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:18:38.180931 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:18:38.180895 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:18:52.240317 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:18:52.240286 2566 scope.go:117] "RemoveContainer" containerID="48c3e82c3a3e3ccde0a3cec1b4b7852c7bfc6b305853950394b103941a107739" Jul 22 19:19:23.854753 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:23.854703 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b"] Jul 22 19:19:23.855274 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:23.855032 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" containerID="cri-o://ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64" gracePeriod=30 Jul 22 19:19:24.900497 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:24.900457 2566 patch_prober.go:28] interesting pod/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:19:24.900887 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:24.900545 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:19:27.492051 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:27.492013 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:19:29.107891 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.107866 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:19:29.124440 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.124407 2566 generic.go:356] "Generic (PLEG): container finished" podID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerID="ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64" exitCode=0 Jul 22 19:19:29.124588 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.124459 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" Jul 22 19:19:29.124588 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.124495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" event={"ID":"1fc10e56-e82b-4a4c-9261-0f2671e9673e","Type":"ContainerDied","Data":"ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64"} Jul 22 19:19:29.124588 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.124559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b" event={"ID":"1fc10e56-e82b-4a4c-9261-0f2671e9673e","Type":"ContainerDied","Data":"1d4f0f9a291d3bda94fe9f70b5d334215a0bc76640bfeb04a70dce18873aa620"} Jul 22 19:19:29.124588 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.124575 2566 scope.go:117] "RemoveContainer" containerID="ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64" Jul 22 19:19:29.136854 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.136832 2566 scope.go:117] "RemoveContainer" containerID="18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7" Jul 22 19:19:29.145534 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.145515 2566 scope.go:117] "RemoveContainer" containerID="ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64" Jul 22 19:19:29.145873 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:19:29.145853 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64\": container with ID starting with ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64 not found: ID does not exist" containerID="ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64" Jul 22 19:19:29.145944 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.145882 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64"} err="failed to get container status \"ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64\": rpc error: code = NotFound desc = could not find container \"ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64\": container with ID starting with ea81991bd7863c9659f05666cb686bba563f5da7df73ce0118614415f75e5b64 not found: ID does not exist" Jul 22 19:19:29.145944 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.145902 2566 scope.go:117] "RemoveContainer" containerID="18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7" Jul 22 19:19:29.146147 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:19:29.146130 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7\": container with ID starting with 18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7 not found: ID does not exist" containerID="18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7" Jul 22 19:19:29.146192 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.146152 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7"} err="failed to get container status \"18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7\": rpc error: code = NotFound desc = could not find container \"18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7\": container with ID starting with 18fdd384cbdd6f32b8d87521bddc0b5f6a760d553cb6992d397bebe1edd342b7 not found: ID does not exist" Jul 22 19:19:29.209247 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209222 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-podinfo\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209390 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209264 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-credential-socket\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209390 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209293 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-certs\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209390 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209349 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-token\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209391 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istiod-ca-cert\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209421 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-data\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209450 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-socket\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209483 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-envoy\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209508 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bm5wg\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-kube-api-access-bm5wg\") pod \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\" (UID: \"1fc10e56-e82b-4a4c-9261-0f2671e9673e\") " Jul 22 19:19:29.209550 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209508 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:19:29.209880 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209726 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:19:29.209942 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209906 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:19:29.209998 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209935 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.209998 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209938 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:19:29.209998 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.209956 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.210365 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.210340 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:19:29.211611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.211580 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:19:29.211709 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.211633 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:19:29.212038 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.212018 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:19:29.212038 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.212031 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-kube-api-access-bm5wg" (OuterVolumeSpecName: "kube-api-access-bm5wg") pod "1fc10e56-e82b-4a4c-9261-0f2671e9673e" (UID: "1fc10e56-e82b-4a4c-9261-0f2671e9673e"). InnerVolumeSpecName "kube-api-access-bm5wg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:19:29.310939 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310892 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.310939 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310933 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.310939 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310942 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.310939 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310950 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.311204 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310959 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm5wg\" (UniqueName: \"kubernetes.io/projected/1fc10e56-e82b-4a4c-9261-0f2671e9673e-kube-api-access-bm5wg\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.311204 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310968 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fc10e56-e82b-4a4c-9261-0f2671e9673e-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.311204 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.310976 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fc10e56-e82b-4a4c-9261-0f2671e9673e-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:19:29.452077 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.452043 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b"] Jul 22 19:19:29.456076 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:29.456051 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-73899c00-openshift-default-5cf96b8cdc-wck6b"] Jul 22 19:19:30.044449 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:30.044406 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" path="/var/lib/kubelet/pods/1fc10e56-e82b-4a4c-9261-0f2671e9673e/volumes" Jul 22 19:19:37.385887 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:37.385845 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:19:52.200804 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:52.200774 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:19:52.207066 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:52.207036 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:19:52.218292 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:52.218270 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:19:52.223796 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:19:52.223781 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:20:19.765140 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.765056 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9"] Jul 22 19:20:19.765585 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.765489 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" Jul 22 19:20:19.765585 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.765504 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" Jul 22 19:20:19.765585 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.765518 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" Jul 22 19:20:19.765585 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.765523 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2faeb50a-5681-4a79-901e-8d7a7b57df39" containerName="istio-proxy" Jul 22 19:20:19.765717 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.765595 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" Jul 22 19:20:19.768605 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.768587 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.771616 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.771597 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c4b608-openshift-default-dockercfg-tntb6\"" Jul 22 19:20:19.797687 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.797661 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9"] Jul 22 19:20:19.866298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istiod-ca-cert\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866477 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-certs\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866477 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-socket\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866477 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpl5n\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-kube-api-access-tpl5n\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866477 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866389 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-credential-socket\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866477 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-data\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866671 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-envoy\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866671 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-token\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.866671 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.866542 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-podinfo\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.967762 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.967723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpl5n\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-kube-api-access-tpl5n\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.967967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.967767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-credential-socket\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.967967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.967804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-data\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.967967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.967921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-envoy\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.967967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.967947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-token\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968194 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.967971 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-podinfo\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istiod-ca-cert\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-certs\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968205 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-credential-socket\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968423 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968265 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-data\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968590 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968426 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-socket\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968668 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968645 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-certs\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968668 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968658 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istiod-ca-cert\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.968830 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.968798 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-socket\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.970310 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.970288 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-envoy\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.970475 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.970456 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-podinfo\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.976849 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.976808 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpl5n\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-kube-api-access-tpl5n\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:19.977943 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:19.977918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-token\") pod \"e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:20.078463 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.078363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:20.237894 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.237861 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9"] Jul 22 19:20:20.241282 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:20:20.241252 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd043e35_2e24_4cb4_a6b1_407756ea1ccd.slice/crio-bbd5bcb2c63ef294b6ef49096e9d90cde8aafb90887e1c7d82885f0403e8c038 WatchSource:0}: Error finding container bbd5bcb2c63ef294b6ef49096e9d90cde8aafb90887e1c7d82885f0403e8c038: Status 404 returned error can't find the container with id bbd5bcb2c63ef294b6ef49096e9d90cde8aafb90887e1c7d82885f0403e8c038 Jul 22 19:20:20.243507 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.243475 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:20.243570 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.243545 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:20.243607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.243578 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:20.400298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.400262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" event={"ID":"dd043e35-2e24-4cb4-a6b1-407756ea1ccd","Type":"ContainerStarted","Data":"920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11"} Jul 22 19:20:20.400488 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.400320 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" event={"ID":"dd043e35-2e24-4cb4-a6b1-407756ea1ccd","Type":"ContainerStarted","Data":"bbd5bcb2c63ef294b6ef49096e9d90cde8aafb90887e1c7d82885f0403e8c038"} Jul 22 19:20:20.421695 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:20.421633 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podStartSLOduration=1.421613315 podStartE2EDuration="1.421613315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:20:20.418652393 +0000 UTC m=+2428.936463307" watchObservedRunningTime="2026-07-22 19:20:20.421613315 +0000 UTC m=+2428.939424241" Jul 22 19:20:21.079023 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:21.078979 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:21.084772 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:21.084741 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:21.405751 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:21.405716 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:21.406765 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:21.406745 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:27.613137 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.613098 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n"] Jul 22 19:20:27.613582 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.613530 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" Jul 22 19:20:27.613582 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.613544 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" Jul 22 19:20:27.613677 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.613623 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc10e56-e82b-4a4c-9261-0f2671e9673e" containerName="istio-proxy" Jul 22 19:20:27.616910 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.616893 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.619449 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.619426 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ba9716-openshift-default-dockercfg-7pft4\"" Jul 22 19:20:27.639557 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.639525 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n"] Jul 22 19:20:27.743270 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04479338-f463-4e7a-a165-bb04bc7faba5-istiod-ca-cert\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743454 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743288 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-socket\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743454 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-envoy\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743454 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-istio-token\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743454 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-credential-socket\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743637 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04479338-f463-4e7a-a165-bb04bc7faba5-istio-podinfo\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743637 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743549 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-certs\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743637 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743612 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg82w\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-kube-api-access-xg82w\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.743637 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.743632 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-data\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.844733 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.844698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04479338-f463-4e7a-a165-bb04bc7faba5-istio-podinfo\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.844930 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.844741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-certs\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.844977 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.844933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg82w\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-kube-api-access-xg82w\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845026 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.844974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-data\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845070 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04479338-f463-4e7a-a165-bb04bc7faba5-istiod-ca-cert\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845083 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-socket\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-certs\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-envoy\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845290 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-istio-token\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845290 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845189 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-credential-socket\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845399 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845367 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-data\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845693 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-credential-socket\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845803 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845676 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-socket\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.845803 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.845742 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04479338-f463-4e7a-a165-bb04bc7faba5-istiod-ca-cert\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.847358 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.847327 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-envoy\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.847557 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.847539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04479338-f463-4e7a-a165-bb04bc7faba5-istio-podinfo\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.852412 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.852387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-istio-token\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.852619 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.852600 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg82w\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-kube-api-access-xg82w\") pod \"e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:27.927682 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:27.927652 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:28.066160 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.066131 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n"] Jul 22 19:20:28.067860 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:20:28.067807 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04479338_f463_4e7a_a165_bb04bc7faba5.slice/crio-e61a6cd160dc4f8973c8512021b7ee01d23ad843cb66800c4dae6d2d343008da WatchSource:0}: Error finding container e61a6cd160dc4f8973c8512021b7ee01d23ad843cb66800c4dae6d2d343008da: Status 404 returned error can't find the container with id e61a6cd160dc4f8973c8512021b7ee01d23ad843cb66800c4dae6d2d343008da Jul 22 19:20:28.069968 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.069933 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:28.070056 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.070009 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:28.070056 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.070042 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:28.449302 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.449267 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" event={"ID":"04479338-f463-4e7a-a165-bb04bc7faba5","Type":"ContainerStarted","Data":"b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3"} Jul 22 19:20:28.449482 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.449322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" event={"ID":"04479338-f463-4e7a-a165-bb04bc7faba5","Type":"ContainerStarted","Data":"e61a6cd160dc4f8973c8512021b7ee01d23ad843cb66800c4dae6d2d343008da"} Jul 22 19:20:28.470028 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.469987 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" podStartSLOduration=1.469971942 podStartE2EDuration="1.469971942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:20:28.468192169 +0000 UTC m=+2436.986003093" watchObservedRunningTime="2026-07-22 19:20:28.469971942 +0000 UTC m=+2436.987782857" Jul 22 19:20:28.927806 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.927770 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:28.986805 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:28.986773 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:20:29.940264 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:29.940227 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:30.459972 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:30.459938 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:30.460760 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:30.460737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:36.673094 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:36.672841 2566 patch_prober.go:28] interesting pod/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:54384->10.133.0.53:15021: read: connection reset by peer" start-of-body= Jul 22 19:20:36.673094 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:36.672924 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": read tcp 10.133.0.2:54384->10.133.0.53:15021: read: connection reset by peer" Jul 22 19:20:36.683135 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:20:36.683107 2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd043e35_2e24_4cb4_a6b1_407756ea1ccd.slice/crio-920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11.scope\": RecentStats: unable to find data in memory cache]" Jul 22 19:20:37.508936 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:37.508903 2566 generic.go:356] "Generic (PLEG): container finished" podID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerID="920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11" exitCode=0 Jul 22 19:20:37.509121 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:37.508977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" event={"ID":"dd043e35-2e24-4cb4-a6b1-407756ea1ccd","Type":"ContainerDied","Data":"920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11"} Jul 22 19:20:37.509424 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:37.509409 2566 scope.go:117] "RemoveContainer" containerID="920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11" Jul 22 19:20:37.510497 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:37.510468 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:37.510584 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:37.510548 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:37.510622 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:37.510607 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:38.517720 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:38.517675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" event={"ID":"dd043e35-2e24-4cb4-a6b1-407756ea1ccd","Type":"ContainerStarted","Data":"482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce"} Jul 22 19:20:39.079215 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:39.079173 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:39.079400 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:39.079368 2566 patch_prober.go:28] interesting pod/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:20:39.079445 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:39.079417 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 19:20:40.079421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:40.079334 2566 patch_prober.go:28] interesting pod/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:20:40.079421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:40.079384 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 19:20:41.079038 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:41.079000 2566 patch_prober.go:28] interesting pod/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:20:41.079223 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:41.079059 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 19:20:41.882372 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:41.882332 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:20:42.078978 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:42.078926 2566 patch_prober.go:28] interesting pod/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 19:20:42.079189 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:42.078987 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 19:20:43.082236 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.082154 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:43.082732 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.082461 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:43.083133 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.083116 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:20:43.555583 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.555553 2566 generic.go:356] "Generic (PLEG): container finished" podID="04479338-f463-4e7a-a165-bb04bc7faba5" containerID="b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3" exitCode=0 Jul 22 19:20:43.555764 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.555625 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" event={"ID":"04479338-f463-4e7a-a165-bb04bc7faba5","Type":"ContainerDied","Data":"b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3"} Jul 22 19:20:43.556154 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.556139 2566 scope.go:117] "RemoveContainer" containerID="b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3" Jul 22 19:20:43.557269 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.557239 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:43.557347 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.557323 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:43.557398 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:43.557369 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:20:44.563359 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:44.563318 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" event={"ID":"04479338-f463-4e7a-a165-bb04bc7faba5","Type":"ContainerStarted","Data":"5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515"} Jul 22 19:20:44.930245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:44.929749 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:44.930245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:44.929912 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 22 19:20:44.930245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:44.929973 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 22 19:20:45.460450 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:45.460401 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:45.928428 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:45.928391 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 22 19:20:45.929032 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:45.928999 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 22 19:20:46.928160 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:46.928110 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 22 19:20:46.928384 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:46.928195 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 22 19:20:47.928959 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:47.928918 2566 patch_prober.go:28] interesting pod/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 22 19:20:47.929552 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:47.929517 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 22 19:20:48.932405 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:48.932368 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:48.933418 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:48.933394 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:20:51.298721 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:20:51.298683 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:21:10.494785 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:21:10.494743 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:22:09.684160 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:09.684122 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:22:21.989271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:21.989234 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:22:27.551860 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:27.551829 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9"] Jul 22 19:22:27.552232 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:27.552074 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" containerID="cri-o://482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce" gracePeriod=30 Jul 22 19:22:28.084586 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:28.084546 2566 patch_prober.go:28] interesting pod/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:22:28.084785 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:28.084644 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:22:32.810790 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.810725 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:22:32.834291 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834261 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-podinfo\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834462 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834320 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-token\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834462 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834363 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-envoy\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834462 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834415 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-socket\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834462 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834451 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-credential-socket\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834693 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834478 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istiod-ca-cert\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834693 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834552 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpl5n\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-kube-api-access-tpl5n\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834693 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834582 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-certs\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.834693 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.834658 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-data\") pod \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\" (UID: \"dd043e35-2e24-4cb4-a6b1-407756ea1ccd\") " Jul 22 19:22:32.835169 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.835044 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:32.835471 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.835418 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.835594 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.835507 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:32.835665 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.835638 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:32.835934 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.835806 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:22:32.836216 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.836193 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:32.837725 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.837493 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:22:32.837725 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.837515 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-kube-api-access-tpl5n" (OuterVolumeSpecName: "kube-api-access-tpl5n") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "kube-api-access-tpl5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:22:32.837725 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.837618 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:32.838273 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.838253 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd043e35-2e24-4cb4-a6b1-407756ea1ccd" (UID: "dd043e35-2e24-4cb4-a6b1-407756ea1ccd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:22:32.936885 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936859 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.936885 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936882 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.937049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936891 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.937049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936899 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.937049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936908 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.937049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936916 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpl5n\" (UniqueName: \"kubernetes.io/projected/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-kube-api-access-tpl5n\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.937049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936925 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:32.937049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:32.936933 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd043e35-2e24-4cb4-a6b1-407756ea1ccd-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:33.185627 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.185595 2566 generic.go:356] "Generic (PLEG): container finished" podID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerID="482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce" exitCode=0 Jul 22 19:22:33.185802 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.185650 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" Jul 22 19:22:33.185802 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.185651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" event={"ID":"dd043e35-2e24-4cb4-a6b1-407756ea1ccd","Type":"ContainerDied","Data":"482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce"} Jul 22 19:22:33.185802 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.185768 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9" event={"ID":"dd043e35-2e24-4cb4-a6b1-407756ea1ccd","Type":"ContainerDied","Data":"bbd5bcb2c63ef294b6ef49096e9d90cde8aafb90887e1c7d82885f0403e8c038"} Jul 22 19:22:33.185802 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.185791 2566 scope.go:117] "RemoveContainer" containerID="482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce" Jul 22 19:22:33.195974 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.195952 2566 scope.go:117] "RemoveContainer" containerID="920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11" Jul 22 19:22:33.204191 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.204175 2566 scope.go:117] "RemoveContainer" containerID="482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce" Jul 22 19:22:33.204451 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:22:33.204430 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce\": container with ID starting with 482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce not found: ID does not exist" containerID="482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce" Jul 22 19:22:33.204516 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.204459 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce"} err="failed to get container status \"482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce\": rpc error: code = NotFound desc = could not find container \"482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce\": container with ID starting with 482e281dd8a66c5e802c86745ed280d1ab0a9bc76ba64fb5d667d66608fcafce not found: ID does not exist" Jul 22 19:22:33.204516 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.204475 2566 scope.go:117] "RemoveContainer" containerID="920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11" Jul 22 19:22:33.204700 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:22:33.204683 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11\": container with ID starting with 920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11 not found: ID does not exist" containerID="920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11" Jul 22 19:22:33.204740 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.204707 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11"} err="failed to get container status \"920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11\": rpc error: code = NotFound desc = could not find container \"920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11\": container with ID starting with 920b239642c9f1fd5123acdcb0a607829efbeeff57e664e868ab7d34c25eca11 not found: ID does not exist" Jul 22 19:22:33.209612 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.209585 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9"] Jul 22 19:22:33.214959 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:33.214937 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c4b608-openshift-default-645c96f76c-8n4q9"] Jul 22 19:22:34.043557 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:34.043527 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" path="/var/lib/kubelet/pods/dd043e35-2e24-4cb4-a6b1-407756ea1ccd/volumes" Jul 22 19:22:41.397764 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:41.397730 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:22:49.446540 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:49.446504 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n"] Jul 22 19:22:49.446951 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:49.446744 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" containerID="cri-o://5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515" gracePeriod=30 Jul 22 19:22:50.386404 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.386361 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg"] Jul 22 19:22:50.386940 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.386919 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" Jul 22 19:22:50.387012 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.386944 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" Jul 22 19:22:50.387091 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.387079 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" Jul 22 19:22:50.387126 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.387098 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" Jul 22 19:22:50.390393 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.390375 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.392597 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.392581 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-6c9526-openshift-default-dockercfg-8p6l9\"" Jul 22 19:22:50.401632 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.401611 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg"] Jul 22 19:22:50.496195 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496154 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-data\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496240 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-token\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496279 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-podinfo\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a43dd03-c483-413f-be3b-f9bedcd51392-istiod-ca-cert\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-socket\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496431 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-certs\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-envoy\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85hbn\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-kube-api-access-85hbn\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.496644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.496581 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-credential-socket\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597289 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-token\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597472 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-podinfo\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597472 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597350 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a43dd03-c483-413f-be3b-f9bedcd51392-istiod-ca-cert\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597472 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-socket\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597472 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597433 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-certs\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597677 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-envoy\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597677 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85hbn\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-kube-api-access-85hbn\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597792 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-credential-socket\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597792 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597734 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-data\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.597950 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597898 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-socket\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.598012 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.597970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-certs\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.598184 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.598144 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-data\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.598319 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.598184 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-credential-socket\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.598319 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.598247 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a43dd03-c483-413f-be3b-f9bedcd51392-istiod-ca-cert\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.599779 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.599754 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-envoy\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.599961 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.599945 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-podinfo\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.605222 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.605193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-token\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.605322 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.605234 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85hbn\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-kube-api-access-85hbn\") pod \"e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.702129 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.702045 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:50.859227 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.859193 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg"] Jul 22 19:22:50.861057 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:22:50.861030 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a43dd03_c483_413f_be3b_f9bedcd51392.slice/crio-2ae692d052c16b6f4cbeae9691f551f8d3aee458cdde835b9d4d8c21208892c7 WatchSource:0}: Error finding container 2ae692d052c16b6f4cbeae9691f551f8d3aee458cdde835b9d4d8c21208892c7: Status 404 returned error can't find the container with id 2ae692d052c16b6f4cbeae9691f551f8d3aee458cdde835b9d4d8c21208892c7 Jul 22 19:22:50.863362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.863320 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:22:50.863430 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.863415 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:22:50.863471 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:50.863459 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:22:51.288600 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:51.288561 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" event={"ID":"9a43dd03-c483-413f-be3b-f9bedcd51392","Type":"ContainerStarted","Data":"7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c"} Jul 22 19:22:51.288769 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:51.288644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" event={"ID":"9a43dd03-c483-413f-be3b-f9bedcd51392","Type":"ContainerStarted","Data":"2ae692d052c16b6f4cbeae9691f551f8d3aee458cdde835b9d4d8c21208892c7"} Jul 22 19:22:51.307664 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:51.307615 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podStartSLOduration=1.3075991089999999 podStartE2EDuration="1.307599109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:22:51.305428282 +0000 UTC m=+2579.823239211" watchObservedRunningTime="2026-07-22 19:22:51.307599109 +0000 UTC m=+2579.825410022" Jul 22 19:22:51.702460 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:51.702426 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:51.707237 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:51.707212 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:52.293576 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:52.293543 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:52.294643 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:52.294606 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:22:54.702012 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.701971 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:22:54.835499 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835424 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xg82w\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-kube-api-access-xg82w\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835499 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835464 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-data\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835507 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-istio-token\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835555 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-socket\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835587 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04479338-f463-4e7a-a165-bb04bc7faba5-istio-podinfo\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835611 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-envoy\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835627 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-credential-socket\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835664 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04479338-f463-4e7a-a165-bb04bc7faba5-istiod-ca-cert\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.835716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835708 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-certs\") pod \"04479338-f463-4e7a-a165-bb04bc7faba5\" (UID: \"04479338-f463-4e7a-a165-bb04bc7faba5\") " Jul 22 19:22:54.836100 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.835903 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:54.836158 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.836115 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.836158 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.836108 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:54.836158 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.836117 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:54.836273 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.836244 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04479338-f463-4e7a-a165-bb04bc7faba5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:22:54.836442 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.836419 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-data" (OuterVolumeSpecName: "istio-data") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:54.837757 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.837729 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-kube-api-access-xg82w" (OuterVolumeSpecName: "kube-api-access-xg82w") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "kube-api-access-xg82w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:22:54.837912 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.837811 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:22:54.838021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.838003 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-istio-token" (OuterVolumeSpecName: "istio-token") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:22:54.838117 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.838097 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04479338-f463-4e7a-a165-bb04bc7faba5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04479338-f463-4e7a-a165-bb04bc7faba5" (UID: "04479338-f463-4e7a-a165-bb04bc7faba5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:22:54.937300 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937276 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937304 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04479338-f463-4e7a-a165-bb04bc7faba5-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937318 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937330 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937344 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04479338-f463-4e7a-a165-bb04bc7faba5-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937357 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937369 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg82w\" (UniqueName: \"kubernetes.io/projected/04479338-f463-4e7a-a165-bb04bc7faba5-kube-api-access-xg82w\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:54.937403 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:54.937381 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04479338-f463-4e7a-a165-bb04bc7faba5-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:22:55.311638 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.311608 2566 generic.go:356] "Generic (PLEG): container finished" podID="04479338-f463-4e7a-a165-bb04bc7faba5" containerID="5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515" exitCode=0 Jul 22 19:22:55.311834 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.311655 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" Jul 22 19:22:55.311834 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.311691 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" event={"ID":"04479338-f463-4e7a-a165-bb04bc7faba5","Type":"ContainerDied","Data":"5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515"} Jul 22 19:22:55.311834 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.311747 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n" event={"ID":"04479338-f463-4e7a-a165-bb04bc7faba5","Type":"ContainerDied","Data":"e61a6cd160dc4f8973c8512021b7ee01d23ad843cb66800c4dae6d2d343008da"} Jul 22 19:22:55.311834 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.311769 2566 scope.go:117] "RemoveContainer" containerID="5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515" Jul 22 19:22:55.321476 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.321458 2566 scope.go:117] "RemoveContainer" containerID="b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3" Jul 22 19:22:55.329620 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.329603 2566 scope.go:117] "RemoveContainer" containerID="5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515" Jul 22 19:22:55.329875 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:22:55.329856 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515\": container with ID starting with 5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515 not found: ID does not exist" containerID="5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515" Jul 22 19:22:55.329962 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.329886 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515"} err="failed to get container status \"5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515\": rpc error: code = NotFound desc = could not find container \"5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515\": container with ID starting with 5b1937a90b38493c77b52a8776df5da687f5cc5431bcf4c699d7b2af19bf9515 not found: ID does not exist" Jul 22 19:22:55.329962 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.329923 2566 scope.go:117] "RemoveContainer" containerID="b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3" Jul 22 19:22:55.330181 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:22:55.330164 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3\": container with ID starting with b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3 not found: ID does not exist" containerID="b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3" Jul 22 19:22:55.330225 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.330187 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3"} err="failed to get container status \"b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3\": rpc error: code = NotFound desc = could not find container \"b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3\": container with ID starting with b6e190c2609a4150b52cad4dd2d0864c2b2aca5fadf5b35cbd5cd94ec0dff9b3 not found: ID does not exist" Jul 22 19:22:55.334803 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.334781 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n"] Jul 22 19:22:55.340724 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:55.340703 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ba9716-openshift-default-7f894484c9-cm24n"] Jul 22 19:22:56.044201 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:56.044166 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" path="/var/lib/kubelet/pods/04479338-f463-4e7a-a165-bb04bc7faba5/volumes" Jul 22 19:22:58.597251 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597217 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff"] Jul 22 19:22:58.597613 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597578 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" Jul 22 19:22:58.597613 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597589 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd043e35-2e24-4cb4-a6b1-407756ea1ccd" containerName="istio-proxy" Jul 22 19:22:58.597613 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597601 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" Jul 22 19:22:58.597613 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597607 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" Jul 22 19:22:58.597746 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597618 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" Jul 22 19:22:58.597746 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597623 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" Jul 22 19:22:58.597746 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.597685 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" Jul 22 19:22:58.602311 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.602289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.604475 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.604459 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-65c717-openshift-default-dockercfg-lld7m\"" Jul 22 19:22:58.612924 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.612893 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff"] Jul 22 19:22:58.773794 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istiod-ca-cert\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.773794 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-credential-socket\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774043 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp5zh\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-kube-api-access-sp5zh\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774043 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-token\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774043 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773940 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-data\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774043 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773962 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-podinfo\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774043 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.773984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-certs\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774203 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.774042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-socket\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.774203 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.774109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-envoy\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875124 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istiod-ca-cert\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875305 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-credential-socket\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875305 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp5zh\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-kube-api-access-sp5zh\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875305 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875177 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-token\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875305 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875203 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-data\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875305 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-podinfo\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875305 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-certs\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-socket\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-envoy\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-credential-socket\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-data\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.875800 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.875677 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-certs\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.876072 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.876046 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istiod-ca-cert\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.876176 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.876128 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-socket\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.877697 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.877666 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-envoy\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.877808 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.877712 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-podinfo\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.883641 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.883614 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-token\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.883868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.883849 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp5zh\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-kube-api-access-sp5zh\") pod \"e2e-api-b-65c717-openshift-default-6574454bf9-p67ff\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:58.912755 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:58.912730 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:22:59.082498 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.082468 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff"] Jul 22 19:22:59.084110 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:22:59.084077 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6af3fc6d_f687_41b8_bc9a_20d5643197f5.slice/crio-0de4c92c61d1321505bdddc8291bf7151c8453be92fcad24c04ad492ad790ce4 WatchSource:0}: Error finding container 0de4c92c61d1321505bdddc8291bf7151c8453be92fcad24c04ad492ad790ce4: Status 404 returned error can't find the container with id 0de4c92c61d1321505bdddc8291bf7151c8453be92fcad24c04ad492ad790ce4 Jul 22 19:22:59.086937 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.086883 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:22:59.087051 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.086986 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:22:59.087108 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.087061 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:22:59.337790 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.337752 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" event={"ID":"6af3fc6d-f687-41b8-bc9a-20d5643197f5","Type":"ContainerStarted","Data":"26723a346c1371c9b9782eab6392467c52b3f4fa7b8a852d11e8fe2082a49499"} Jul 22 19:22:59.338265 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.337830 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" event={"ID":"6af3fc6d-f687-41b8-bc9a-20d5643197f5","Type":"ContainerStarted","Data":"0de4c92c61d1321505bdddc8291bf7151c8453be92fcad24c04ad492ad790ce4"} Jul 22 19:22:59.356049 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.355983 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" podStartSLOduration=1.355958828 podStartE2EDuration="1.355958828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:22:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:22:59.354419729 +0000 UTC m=+2587.872230635" watchObservedRunningTime="2026-07-22 19:22:59.355958828 +0000 UTC m=+2587.873769742" Jul 22 19:22:59.912856 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:22:59.912811 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:00.397362 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:00.397324 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:23:00.918901 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:00.918865 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:01.351199 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:01.351116 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:01.352213 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:01.352197 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:08.335116 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:08.335033 2566 patch_prober.go:28] interesting pod/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 19:23:08.335784 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:08.335745 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 19:23:09.395991 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:09.395957 2566 generic.go:356] "Generic (PLEG): container finished" podID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerID="7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c" exitCode=0 Jul 22 19:23:09.396409 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:09.396032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" event={"ID":"9a43dd03-c483-413f-be3b-f9bedcd51392","Type":"ContainerDied","Data":"7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c"} Jul 22 19:23:09.396459 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:09.396421 2566 scope.go:117] "RemoveContainer" containerID="7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c" Jul 22 19:23:09.397426 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:09.397398 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:09.397492 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:09.397468 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:09.397530 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:09.397500 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:10.404263 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:10.404225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" event={"ID":"9a43dd03-c483-413f-be3b-f9bedcd51392","Type":"ContainerStarted","Data":"b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443"} Jul 22 19:23:10.703084 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:10.702986 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:23:10.703301 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:10.703278 2566 patch_prober.go:28] interesting pod/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 19:23:10.703353 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:10.703332 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 19:23:11.703214 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:11.703173 2566 patch_prober.go:28] interesting pod/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 19:23:11.703627 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:11.703246 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 19:23:11.896271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:11.896228 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:23:12.702978 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:12.702934 2566 patch_prober.go:28] interesting pod/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 19:23:12.703164 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:12.703009 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 19:23:13.702835 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:13.702774 2566 patch_prober.go:28] interesting pod/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 19:23:13.703291 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:13.702875 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 19:23:14.705911 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:14.705806 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:23:14.706294 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:14.706198 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:23:14.707342 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:14.707322 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:23:15.439256 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:15.439186 2566 generic.go:356] "Generic (PLEG): container finished" podID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerID="26723a346c1371c9b9782eab6392467c52b3f4fa7b8a852d11e8fe2082a49499" exitCode=0 Jul 22 19:23:15.439429 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:15.439259 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" event={"ID":"6af3fc6d-f687-41b8-bc9a-20d5643197f5","Type":"ContainerDied","Data":"26723a346c1371c9b9782eab6392467c52b3f4fa7b8a852d11e8fe2082a49499"} Jul 22 19:23:15.439775 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:15.439759 2566 scope.go:117] "RemoveContainer" containerID="26723a346c1371c9b9782eab6392467c52b3f4fa7b8a852d11e8fe2082a49499" Jul 22 19:23:15.440740 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:15.440715 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:15.440797 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:15.440782 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:15.440854 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:15.440842 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:23:16.352109 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:16.352066 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:16.446984 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:16.446945 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" event={"ID":"6af3fc6d-f687-41b8-bc9a-20d5643197f5","Type":"ContainerStarted","Data":"eb6ee00489761aaca826d380c4d180a1621ab6c1bc0ad8189413a2caf40d2def"} Jul 22 19:23:16.913911 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:16.913867 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:16.915104 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:16.915070 2566 patch_prober.go:28] interesting pod/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:23:16.915356 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:16.915325 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 19:23:17.913783 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:17.913748 2566 patch_prober.go:28] interesting pod/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:23:17.914346 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:17.913805 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 19:23:18.913530 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:18.913494 2566 patch_prober.go:28] interesting pod/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:23:18.913706 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:18.913555 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 19:23:19.914056 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:19.914016 2566 patch_prober.go:28] interesting pod/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 19:23:19.914558 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:19.914086 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 19:23:20.917549 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:20.917514 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:20.918048 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:20.917947 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:20.918556 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:20.918538 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:23:32.587355 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:32.587319 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:23:40.582354 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:40.582318 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg"] Jul 22 19:23:40.582917 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:40.582574 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" containerID="cri-o://b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443" gracePeriod=30 Jul 22 19:23:44.708451 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:44.708412 2566 patch_prober.go:28] interesting pod/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 19:23:44.708941 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:44.708492 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 19:23:45.842026 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.841963 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:23:45.938775 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.938741 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85hbn\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-kube-api-access-85hbn\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.938956 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.938798 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-credential-socket\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.938956 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.938846 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-podinfo\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.938956 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.938906 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-certs\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.939136 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.938961 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-envoy\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.939136 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.938986 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-data\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.939136 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939031 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-token\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.939136 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939064 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a43dd03-c483-413f-be3b-f9bedcd51392-istiod-ca-cert\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.939136 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939098 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-socket\") pod \"9a43dd03-c483-413f-be3b-f9bedcd51392\" (UID: \"9a43dd03-c483-413f-be3b-f9bedcd51392\") " Jul 22 19:23:45.939389 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939192 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:45.939389 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939295 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:45.939501 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939442 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:45.939660 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939624 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a43dd03-c483-413f-be3b-f9bedcd51392-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:23:45.939795 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939697 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:45.939795 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939719 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:45.939795 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.939732 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:45.940124 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.940096 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-data" (OuterVolumeSpecName: "istio-data") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:45.941303 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.941270 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:23:45.941303 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.941290 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-kube-api-access-85hbn" (OuterVolumeSpecName: "kube-api-access-85hbn") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "kube-api-access-85hbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:23:45.941604 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.941574 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:23:45.941705 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:45.941581 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a43dd03-c483-413f-be3b-f9bedcd51392" (UID: "9a43dd03-c483-413f-be3b-f9bedcd51392"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:23:46.041021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.040986 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85hbn\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-kube-api-access-85hbn\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:46.041021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.041021 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:46.041245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.041036 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:46.041245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.041049 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:46.041245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.041061 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a43dd03-c483-413f-be3b-f9bedcd51392-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:46.041245 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.041073 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a43dd03-c483-413f-be3b-f9bedcd51392-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:23:46.622204 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.622170 2566 generic.go:356] "Generic (PLEG): container finished" podID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerID="b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443" exitCode=0 Jul 22 19:23:46.622380 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.622224 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" Jul 22 19:23:46.622380 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.622265 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" event={"ID":"9a43dd03-c483-413f-be3b-f9bedcd51392","Type":"ContainerDied","Data":"b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443"} Jul 22 19:23:46.622380 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.622328 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg" event={"ID":"9a43dd03-c483-413f-be3b-f9bedcd51392","Type":"ContainerDied","Data":"2ae692d052c16b6f4cbeae9691f551f8d3aee458cdde835b9d4d8c21208892c7"} Jul 22 19:23:46.622380 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.622355 2566 scope.go:117] "RemoveContainer" containerID="b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443" Jul 22 19:23:46.631760 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.631742 2566 scope.go:117] "RemoveContainer" containerID="7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c" Jul 22 19:23:46.639522 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.639485 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg"] Jul 22 19:23:46.640495 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.640315 2566 scope.go:117] "RemoveContainer" containerID="b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443" Jul 22 19:23:46.640731 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:23:46.640636 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443\": container with ID starting with b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443 not found: ID does not exist" containerID="b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443" Jul 22 19:23:46.640731 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.640667 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443"} err="failed to get container status \"b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443\": rpc error: code = NotFound desc = could not find container \"b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443\": container with ID starting with b8e569e9e49d8ed5271a78486a4119f42a726ad37c142634940b7d0883bbe443 not found: ID does not exist" Jul 22 19:23:46.640731 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.640684 2566 scope.go:117] "RemoveContainer" containerID="7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c" Jul 22 19:23:46.641024 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:23:46.640974 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c\": container with ID starting with 7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c not found: ID does not exist" containerID="7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c" Jul 22 19:23:46.641024 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.641003 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c"} err="failed to get container status \"7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c\": rpc error: code = NotFound desc = could not find container \"7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c\": container with ID starting with 7276c9a18358287ffcef361c4798f873f8a5c02939b9f0359333f0e8f644104c not found: ID does not exist" Jul 22 19:23:46.643034 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:46.643013 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-6c9526-openshift-default-6794bcff65-64bqg"] Jul 22 19:23:48.043978 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:23:48.043937 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" path="/var/lib/kubelet/pods/9a43dd03-c483-413f-be3b-f9bedcd51392/volumes" Jul 22 19:24:01.493036 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:01.493000 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:24:07.633437 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:07.633403 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff"] Jul 22 19:24:07.633969 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:07.633613 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" containerID="cri-o://eb6ee00489761aaca826d380c4d180a1621ab6c1bc0ad8189413a2caf40d2def" gracePeriod=30 Jul 22 19:24:08.567532 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.567481 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq"] Jul 22 19:24:08.568144 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568115 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" Jul 22 19:24:08.568144 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568143 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" Jul 22 19:24:08.568297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568162 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" Jul 22 19:24:08.568297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568171 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" Jul 22 19:24:08.568297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568264 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" Jul 22 19:24:08.568297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568278 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a43dd03-c483-413f-be3b-f9bedcd51392" containerName="istio-proxy" Jul 22 19:24:08.568297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.568288 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="04479338-f463-4e7a-a165-bb04bc7faba5" containerName="istio-proxy" Jul 22 19:24:08.571846 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.571805 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.574021 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.574003 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-4ac498-openshift-default-dockercfg-r469v\"" Jul 22 19:24:08.582094 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.582072 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq"] Jul 22 19:24:08.646391 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-token\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-envoy\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-podinfo\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646514 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntqv7\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-kube-api-access-ntqv7\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646565 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-socket\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-certs\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-credential-socket\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/341f6a1c-dde7-4165-9f2c-e83f3066f344-istiod-ca-cert\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.646698 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.646700 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-data\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747773 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-token\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-envoy\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747811 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-podinfo\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntqv7\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-kube-api-access-ntqv7\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-socket\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747911 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-certs\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.747947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747937 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-credential-socket\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748261 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.747978 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/341f6a1c-dde7-4165-9f2c-e83f3066f344-istiod-ca-cert\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748261 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.748021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-data\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.748317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-socket\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748374 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.748344 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-data\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748465 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.748428 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-certs\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748465 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.748451 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-credential-socket\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.748738 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.748718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/341f6a1c-dde7-4165-9f2c-e83f3066f344-istiod-ca-cert\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.750106 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.750083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-envoy\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.750480 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.750446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-podinfo\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.755383 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.755358 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-token\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.755580 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.755563 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntqv7\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-kube-api-access-ntqv7\") pod \"e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:08.882896 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:08.882851 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:09.016293 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.016262 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq"] Jul 22 19:24:09.017782 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:24:09.017755 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341f6a1c_dde7_4165_9f2c_e83f3066f344.slice/crio-69b4880a42800a06c109412fcba97ffea5f3218096f206e13dbd0c31b0efddc5 WatchSource:0}: Error finding container 69b4880a42800a06c109412fcba97ffea5f3218096f206e13dbd0c31b0efddc5: Status 404 returned error can't find the container with id 69b4880a42800a06c109412fcba97ffea5f3218096f206e13dbd0c31b0efddc5 Jul 22 19:24:09.019989 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.019955 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:09.020092 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.020020 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:09.020092 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.020051 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:09.749767 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.749726 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" event={"ID":"341f6a1c-dde7-4165-9f2c-e83f3066f344","Type":"ContainerStarted","Data":"18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb"} Jul 22 19:24:09.750164 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.749788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" event={"ID":"341f6a1c-dde7-4165-9f2c-e83f3066f344","Type":"ContainerStarted","Data":"69b4880a42800a06c109412fcba97ffea5f3218096f206e13dbd0c31b0efddc5"} Jul 22 19:24:09.770601 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.770550 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podStartSLOduration=1.7705353449999999 podStartE2EDuration="1.770535345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:09.767502019 +0000 UTC m=+2658.285312971" watchObservedRunningTime="2026-07-22 19:24:09.770535345 +0000 UTC m=+2658.288346260" Jul 22 19:24:09.883578 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:09.883542 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:10.888676 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:10.888646 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:11.761727 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:11.761698 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:11.762714 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:11.762693 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:12.769259 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.769226 2566 generic.go:356] "Generic (PLEG): container finished" podID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerID="eb6ee00489761aaca826d380c4d180a1621ab6c1bc0ad8189413a2caf40d2def" exitCode=0 Jul 22 19:24:12.769742 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.769297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" event={"ID":"6af3fc6d-f687-41b8-bc9a-20d5643197f5","Type":"ContainerDied","Data":"eb6ee00489761aaca826d380c4d180a1621ab6c1bc0ad8189413a2caf40d2def"} Jul 22 19:24:12.769742 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.769374 2566 scope.go:117] "RemoveContainer" containerID="26723a346c1371c9b9782eab6392467c52b3f4fa7b8a852d11e8fe2082a49499" Jul 22 19:24:12.893584 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.893519 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:24:12.992487 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992454 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istiod-ca-cert\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992524 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-certs\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992583 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-data\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992613 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp5zh\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-kube-api-access-sp5zh\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992639 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-podinfo\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992669 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-envoy\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992728 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-socket\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992780 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-token\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992859 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-credential-socket\") pod \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\" (UID: \"6af3fc6d-f687-41b8-bc9a-20d5643197f5\") " Jul 22 19:24:12.992967 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.992905 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:12.993171 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.993014 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:24:12.993244 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.993204 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:12.993244 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.993212 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:12.993244 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.993230 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:12.993397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.993346 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:12.993735 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.993707 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:12.995108 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.995084 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-kube-api-access-sp5zh" (OuterVolumeSpecName: "kube-api-access-sp5zh") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "kube-api-access-sp5zh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:12.995215 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.995133 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:24:12.995215 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.995175 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:24:12.995215 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:12.995196 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6af3fc6d-f687-41b8-bc9a-20d5643197f5" (UID: "6af3fc6d-f687-41b8-bc9a-20d5643197f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:24:13.093751 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093726 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.093751 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093751 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp5zh\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-kube-api-access-sp5zh\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.093952 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093762 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.093952 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093770 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.093952 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093779 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.093952 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093786 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6af3fc6d-f687-41b8-bc9a-20d5643197f5-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.093952 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.093796 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6af3fc6d-f687-41b8-bc9a-20d5643197f5-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:24:13.776011 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.775981 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" Jul 22 19:24:13.776421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.776028 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff" event={"ID":"6af3fc6d-f687-41b8-bc9a-20d5643197f5","Type":"ContainerDied","Data":"0de4c92c61d1321505bdddc8291bf7151c8453be92fcad24c04ad492ad790ce4"} Jul 22 19:24:13.776421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.776098 2566 scope.go:117] "RemoveContainer" containerID="eb6ee00489761aaca826d380c4d180a1621ab6c1bc0ad8189413a2caf40d2def" Jul 22 19:24:13.799677 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.799644 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff"] Jul 22 19:24:13.804154 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:13.804126 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-65c717-openshift-default-6574454bf9-p67ff"] Jul 22 19:24:14.044375 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:14.044294 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" path="/var/lib/kubelet/pods/6af3fc6d-f687-41b8-bc9a-20d5643197f5/volumes" Jul 22 19:24:16.948112 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.948080 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f"] Jul 22 19:24:16.948561 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.948491 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" Jul 22 19:24:16.948561 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.948506 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" Jul 22 19:24:16.948675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.948586 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" Jul 22 19:24:16.948675 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.948601 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" Jul 22 19:24:16.953265 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.953241 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:16.955425 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.955404 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-8f2692-openshift-default-dockercfg-kkhz6\"" Jul 22 19:24:16.964160 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:16.964131 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f"] Jul 22 19:24:17.032140 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttg97\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-kube-api-access-ttg97\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032340 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-credential-socket\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032340 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-certs\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032340 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-socket\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032340 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032337 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e512eef-42aa-4f79-b483-c9f7886041ae-istiod-ca-cert\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032478 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032363 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-token\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032478 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-envoy\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032478 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-podinfo\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.032478 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.032457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-data\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.133951 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.133916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttg97\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-kube-api-access-ttg97\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134134 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.133995 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-credential-socket\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134134 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134043 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-certs\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134134 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134094 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-socket\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134134 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e512eef-42aa-4f79-b483-c9f7886041ae-istiod-ca-cert\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134345 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-token\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134345 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-envoy\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134345 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-podinfo\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134345 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-data\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134555 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134530 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-credential-socket\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134618 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134559 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-certs\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134618 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134590 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-socket\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134686 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134635 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-data\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.134961 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.134935 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e512eef-42aa-4f79-b483-c9f7886041ae-istiod-ca-cert\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.136680 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.136648 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-envoy\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.136872 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.136854 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-podinfo\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.141607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.141582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-token\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.141719 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.141681 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttg97\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-kube-api-access-ttg97\") pod \"e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.264607 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.264554 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:17.401901 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.401876 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f"] Jul 22 19:24:17.408032 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.407992 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:17.408126 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.408083 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:17.408166 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.408127 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:17.801060 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.801025 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" event={"ID":"7e512eef-42aa-4f79-b483-c9f7886041ae","Type":"ContainerStarted","Data":"5e2838353a6863576786d0742a4cb2e7c73e4ce62c9fc17268408d6f903af2ea"} Jul 22 19:24:17.801261 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.801089 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" event={"ID":"7e512eef-42aa-4f79-b483-c9f7886041ae","Type":"ContainerStarted","Data":"05abb7717dd6e061abbb4646d4054f65ed49f46dc7860b539ab408bb4da02d70"} Jul 22 19:24:17.820111 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:17.820063 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podStartSLOduration=1.8200498120000002 podStartE2EDuration="1.820049812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 19:24:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:24:17.818032404 +0000 UTC m=+2666.335843337" watchObservedRunningTime="2026-07-22 19:24:17.820049812 +0000 UTC m=+2666.337860725" Jul 22 19:24:18.264983 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:18.264946 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:18.269709 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:18.269685 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:18.807410 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:18.807375 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:18.808406 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:18.808387 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:19.597720 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:19.597682 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:24:27.615297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.615264 2566 patch_prober.go:28] interesting pod/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:48528->10.133.0.57:15021: read: connection reset by peer" start-of-body= Jul 22 19:24:27.615806 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.615765 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:48528->10.133.0.57:15021: read: connection reset by peer" Jul 22 19:24:27.870803 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.870763 2566 generic.go:356] "Generic (PLEG): container finished" podID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerID="18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb" exitCode=0 Jul 22 19:24:27.870977 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.870846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" event={"ID":"341f6a1c-dde7-4165-9f2c-e83f3066f344","Type":"ContainerDied","Data":"18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb"} Jul 22 19:24:27.871274 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.871258 2566 scope.go:117] "RemoveContainer" containerID="18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb" Jul 22 19:24:27.872253 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.872227 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:27.872326 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.872296 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:27.872366 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:27.872326 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:28.878202 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:28.878169 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" event={"ID":"341f6a1c-dde7-4165-9f2c-e83f3066f344","Type":"ContainerStarted","Data":"5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6"} Jul 22 19:24:28.882977 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:28.882960 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:28.883145 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:28.883124 2566 patch_prober.go:28] interesting pod/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:24:28.883196 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:28.883179 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 19:24:29.883421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:29.883383 2566 patch_prober.go:28] interesting pod/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:24:29.883776 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:29.883449 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 19:24:30.192808 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:30.192734 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:24:30.883409 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:30.883370 2566 patch_prober.go:28] interesting pod/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:24:30.883593 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:30.883430 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 19:24:31.883485 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:31.883443 2566 patch_prober.go:28] interesting pod/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 22 19:24:31.883686 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:31.883511 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 22 19:24:32.886908 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:32.886871 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:32.887323 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:32.887280 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:32.887957 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:32.887920 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:24:33.809220 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.809187 2566 patch_prober.go:28] interesting pod/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 19:24:33.809414 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.809261 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 19:24:33.905779 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.905748 2566 generic.go:356] "Generic (PLEG): container finished" podID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerID="5e2838353a6863576786d0742a4cb2e7c73e4ce62c9fc17268408d6f903af2ea" exitCode=0 Jul 22 19:24:33.906144 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.905842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" event={"ID":"7e512eef-42aa-4f79-b483-c9f7886041ae","Type":"ContainerDied","Data":"5e2838353a6863576786d0742a4cb2e7c73e4ce62c9fc17268408d6f903af2ea"} Jul 22 19:24:33.906365 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.906345 2566 scope.go:117] "RemoveContainer" containerID="5e2838353a6863576786d0742a4cb2e7c73e4ce62c9fc17268408d6f903af2ea" Jul 22 19:24:33.907421 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.907389 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:33.907522 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.907471 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:33.907586 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:33.907518 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 22 19:24:34.914579 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:34.914542 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" event={"ID":"7e512eef-42aa-4f79-b483-c9f7886041ae","Type":"ContainerStarted","Data":"cd23afd043323d1706c50620413fd5058558781159ba41d457466579b5858410"} Jul 22 19:24:35.265271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:35.265171 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:35.265452 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:35.265357 2566 patch_prober.go:28] interesting pod/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 19:24:35.265452 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:35.265416 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 19:24:36.265596 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:36.265561 2566 patch_prober.go:28] interesting pod/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 19:24:36.265997 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:36.265620 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 19:24:37.264961 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:37.264924 2566 patch_prober.go:28] interesting pod/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 19:24:37.265138 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:37.264986 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 19:24:38.265894 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:38.265853 2566 patch_prober.go:28] interesting pod/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 19:24:38.266279 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:38.265937 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 19:24:39.268977 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:39.268947 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:39.269451 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:39.269341 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:39.269957 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:39.269937 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:24:48.930978 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:48.930943 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:24:52.237367 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:52.237336 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:24:52.243271 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:52.243248 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:24:52.257321 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:52.257301 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:24:52.262832 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:24:52.262799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:25:19.000163 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:25:19.000121 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:25:32.796739 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:25:32.796702 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:25:48.386462 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:25:48.386426 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:26:07.685376 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:26:07.685340 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:26:26.993119 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:26:26.993034 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:26:46.000450 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:26:46.000415 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:27:03.285208 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:27:03.285175 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:27:21.982175 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:27:21.982136 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:27:32.217480 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:27:32.217440 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:27:57.285102 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:27:57.285014 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:28:30.086556 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:28:30.086516 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:28:49.092356 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:28:49.092318 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:29:07.087542 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:07.087509 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:29:21.286738 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:21.286648 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:29:48.086763 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:48.086724 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:29:52.281388 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:52.281355 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:29:52.287320 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:52.287294 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:29:52.295904 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:52.295885 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:29:52.301074 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:29:52.301056 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:30:04.888780 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:04.888740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:30:19.692645 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:19.692601 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:30:31.947404 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.947367 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7789-140041791079088"] Jul 22 19:30:31.947843 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.947754 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" Jul 22 19:30:31.947843 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.947764 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6af3fc6d-f687-41b8-bc9a-20d5643197f5" containerName="istio-proxy" Jul 22 19:30:31.950989 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.950970 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:31.953176 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.953154 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m4ctx\"" Jul 22 19:30:31.957261 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.957233 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7789-140041791079088"] Jul 22 19:30:31.981843 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:31.981800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6x8w\" (UniqueName: \"kubernetes.io/projected/6953172d-3592-40ae-b5d7-8180cc48567b-kube-api-access-t6x8w\") pod \"test-curl-7789-140041791079088\" (UID: \"6953172d-3592-40ae-b5d7-8180cc48567b\") " pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:32.082662 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:32.082624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6x8w\" (UniqueName: \"kubernetes.io/projected/6953172d-3592-40ae-b5d7-8180cc48567b-kube-api-access-t6x8w\") pod \"test-curl-7789-140041791079088\" (UID: \"6953172d-3592-40ae-b5d7-8180cc48567b\") " pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:32.090337 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:32.090307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6x8w\" (UniqueName: \"kubernetes.io/projected/6953172d-3592-40ae-b5d7-8180cc48567b-kube-api-access-t6x8w\") pod \"test-curl-7789-140041791079088\" (UID: \"6953172d-3592-40ae-b5d7-8180cc48567b\") " pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:32.261689 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:32.261596 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:32.602254 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:32.602224 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7789-140041791079088"] Jul 22 19:30:32.604416 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:30:32.604389 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6953172d_3592_40ae_b5d7_8180cc48567b.slice/crio-29b25f483d3645e02a26b7e21c3c2c0f580500ee65bfc99a85d79bc8e27bc45a WatchSource:0}: Error finding container 29b25f483d3645e02a26b7e21c3c2c0f580500ee65bfc99a85d79bc8e27bc45a: Status 404 returned error can't find the container with id 29b25f483d3645e02a26b7e21c3c2c0f580500ee65bfc99a85d79bc8e27bc45a Jul 22 19:30:32.605947 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:32.605927 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 19:30:32.884965 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:32.884930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7789-140041791079088" event={"ID":"6953172d-3592-40ae-b5d7-8180cc48567b","Type":"ContainerStarted","Data":"29b25f483d3645e02a26b7e21c3c2c0f580500ee65bfc99a85d79bc8e27bc45a"} Jul 22 19:30:33.891988 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:33.891952 2566 generic.go:356] "Generic (PLEG): container finished" podID="6953172d-3592-40ae-b5d7-8180cc48567b" containerID="b2b235942456885075e92543f7ab7d578111a013f2e43319f75f92558ed09100" exitCode=0 Jul 22 19:30:33.892391 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:33.892037 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7789-140041791079088" event={"ID":"6953172d-3592-40ae-b5d7-8180cc48567b","Type":"ContainerDied","Data":"b2b235942456885075e92543f7ab7d578111a013f2e43319f75f92558ed09100"} Jul 22 19:30:35.041748 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.041719 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:35.074524 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.074496 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7789-140041791079088_6953172d-3592-40ae-b5d7-8180cc48567b/test-curl-7789-140041791079088/0.log" Jul 22 19:30:35.096171 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.096132 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7789-140041791079088"] Jul 22 19:30:35.099716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.099690 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7789-140041791079088"] Jul 22 19:30:35.114470 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.114440 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6x8w\" (UniqueName: \"kubernetes.io/projected/6953172d-3592-40ae-b5d7-8180cc48567b-kube-api-access-t6x8w\") pod \"6953172d-3592-40ae-b5d7-8180cc48567b\" (UID: \"6953172d-3592-40ae-b5d7-8180cc48567b\") " Jul 22 19:30:35.116759 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.116726 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6953172d-3592-40ae-b5d7-8180cc48567b-kube-api-access-t6x8w" (OuterVolumeSpecName: "kube-api-access-t6x8w") pod "6953172d-3592-40ae-b5d7-8180cc48567b" (UID: "6953172d-3592-40ae-b5d7-8180cc48567b"). InnerVolumeSpecName "kube-api-access-t6x8w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:30:35.216584 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.216492 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6x8w\" (UniqueName: \"kubernetes.io/projected/6953172d-3592-40ae-b5d7-8180cc48567b-kube-api-access-t6x8w\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:30:35.905327 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.905299 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:35.905480 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:35.905347 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29b25f483d3645e02a26b7e21c3c2c0f580500ee65bfc99a85d79bc8e27bc45a" Jul 22 19:30:35.915932 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:30:35.915904 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7789-140041791079088\" is forbidden: User \"system:node:ip-10-0-137-148.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-148.ec2.internal' and this object" podUID="6953172d-3592-40ae-b5d7-8180cc48567b" pod="opendatahub/test-curl-7789-140041791079088" Jul 22 19:30:36.044002 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:36.043962 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6953172d-3592-40ae-b5d7-8180cc48567b" path="/var/lib/kubelet/pods/6953172d-3592-40ae-b5d7-8180cc48567b/volumes" Jul 22 19:30:37.379568 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:30:37.379529 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:31:01.090402 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:01.090315 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:31:09.258296 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:09.258256 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq"] Jul 22 19:31:09.258761 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:09.258489 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" containerID="cri-o://5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6" gracePeriod=30 Jul 22 19:31:14.520009 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.519951 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:31:14.600661 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600631 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-envoy\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.600868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600675 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-podinfo\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.600868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600707 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-data\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.600868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600739 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-certs\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.600868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600763 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-token\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.600868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600796 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntqv7\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-kube-api-access-ntqv7\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.600868 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600846 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-socket\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.601214 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600938 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/341f6a1c-dde7-4165-9f2c-e83f3066f344-istiod-ca-cert\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.601214 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.600970 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-credential-socket\") pod \"341f6a1c-dde7-4165-9f2c-e83f3066f344\" (UID: \"341f6a1c-dde7-4165-9f2c-e83f3066f344\") " Jul 22 19:31:14.601214 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601107 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:14.601214 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601170 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:14.601439 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601342 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:14.601439 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601393 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.601439 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601413 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.601593 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601474 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/341f6a1c-dde7-4165-9f2c-e83f3066f344-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:31:14.601689 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.601667 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-data" (OuterVolumeSpecName: "istio-data") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:14.603278 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.603253 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-kube-api-access-ntqv7" (OuterVolumeSpecName: "kube-api-access-ntqv7") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "kube-api-access-ntqv7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:31:14.603422 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.603331 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-token" (OuterVolumeSpecName: "istio-token") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:31:14.603422 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.603416 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:31:14.603536 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.603517 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "341f6a1c-dde7-4165-9f2c-e83f3066f344" (UID: "341f6a1c-dde7-4165-9f2c-e83f3066f344"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:14.702233 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702185 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.702233 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702226 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.702233 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702235 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.702233 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702244 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.702504 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702254 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntqv7\" (UniqueName: \"kubernetes.io/projected/341f6a1c-dde7-4165-9f2c-e83f3066f344-kube-api-access-ntqv7\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.702504 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702265 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/341f6a1c-dde7-4165-9f2c-e83f3066f344-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:14.702504 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:14.702273 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/341f6a1c-dde7-4165-9f2c-e83f3066f344-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:15.134188 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.134152 2566 generic.go:356] "Generic (PLEG): container finished" podID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerID="5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6" exitCode=0 Jul 22 19:31:15.134397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.134235 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" Jul 22 19:31:15.134397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.134244 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" event={"ID":"341f6a1c-dde7-4165-9f2c-e83f3066f344","Type":"ContainerDied","Data":"5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6"} Jul 22 19:31:15.134397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.134304 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq" event={"ID":"341f6a1c-dde7-4165-9f2c-e83f3066f344","Type":"ContainerDied","Data":"69b4880a42800a06c109412fcba97ffea5f3218096f206e13dbd0c31b0efddc5"} Jul 22 19:31:15.134397 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.134328 2566 scope.go:117] "RemoveContainer" containerID="5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6" Jul 22 19:31:15.144502 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.144470 2566 scope.go:117] "RemoveContainer" containerID="18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb" Jul 22 19:31:15.152601 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.152582 2566 scope.go:117] "RemoveContainer" containerID="5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6" Jul 22 19:31:15.152859 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:31:15.152839 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6\": container with ID starting with 5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6 not found: ID does not exist" containerID="5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6" Jul 22 19:31:15.152913 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.152867 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6"} err="failed to get container status \"5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6\": rpc error: code = NotFound desc = could not find container \"5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6\": container with ID starting with 5e33dca4dffe0e197331c9f0d82d286122fb4c24af42cc200ebf1e9a7ba976e6 not found: ID does not exist" Jul 22 19:31:15.152913 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.152884 2566 scope.go:117] "RemoveContainer" containerID="18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb" Jul 22 19:31:15.153151 ip-10-0-137-148 kubenswrapper[2566]: E0722 19:31:15.153133 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb\": container with ID starting with 18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb not found: ID does not exist" containerID="18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb" Jul 22 19:31:15.153199 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.153156 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb"} err="failed to get container status \"18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb\": rpc error: code = NotFound desc = could not find container \"18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb\": container with ID starting with 18ad58427ba084cd72f63c8fe3bea7fa22086405e30c6f5f7164ef316e51cedb not found: ID does not exist" Jul 22 19:31:15.156710 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.156688 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq"] Jul 22 19:31:15.160153 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:15.160134 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-4ac498-openshift-default-6c6dd4cd4c-tg7mq"] Jul 22 19:31:16.044104 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:16.044071 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" path="/var/lib/kubelet/pods/341f6a1c-dde7-4165-9f2c-e83f3066f344/volumes" Jul 22 19:31:22.978959 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:22.978922 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-54bq2"] Jul 22 19:31:31.124203 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:31.124149 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f"] Jul 22 19:31:31.124720 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:31.124474 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" containerID="cri-o://cd23afd043323d1706c50620413fd5058558781159ba41d457466579b5858410" gracePeriod=30 Jul 22 19:31:36.253589 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.253555 2566 generic.go:356] "Generic (PLEG): container finished" podID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerID="cd23afd043323d1706c50620413fd5058558781159ba41d457466579b5858410" exitCode=0 Jul 22 19:31:36.254129 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.253620 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" event={"ID":"7e512eef-42aa-4f79-b483-c9f7886041ae","Type":"ContainerDied","Data":"cd23afd043323d1706c50620413fd5058558781159ba41d457466579b5858410"} Jul 22 19:31:36.254129 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.253691 2566 scope.go:117] "RemoveContainer" containerID="5e2838353a6863576786d0742a4cb2e7c73e4ce62c9fc17268408d6f903af2ea" Jul 22 19:31:36.372096 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.372067 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:31:36.406391 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406323 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-certs\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406391 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406378 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-data\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406407 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-envoy\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406434 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-socket\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406505 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-token\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406531 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-credential-socket\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406565 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttg97\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-kube-api-access-ttg97\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406611 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406596 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-podinfo\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406948 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406628 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e512eef-42aa-4f79-b483-c9f7886041ae-istiod-ca-cert\") pod \"7e512eef-42aa-4f79-b483-c9f7886041ae\" (UID: \"7e512eef-42aa-4f79-b483-c9f7886041ae\") " Jul 22 19:31:36.406948 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406617 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:36.406948 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.406791 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:36.407105 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.407032 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.407105 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.407052 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.407220 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.407130 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:36.407220 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.407169 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e512eef-42aa-4f79-b483-c9f7886041ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 19:31:36.407342 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.407267 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:36.409016 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.408985 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:31:36.409126 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.409014 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 19:31:36.409218 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.409201 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 19:31:36.409298 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.409283 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-kube-api-access-ttg97" (OuterVolumeSpecName: "kube-api-access-ttg97") pod "7e512eef-42aa-4f79-b483-c9f7886041ae" (UID: "7e512eef-42aa-4f79-b483-c9f7886041ae"). InnerVolumeSpecName "kube-api-access-ttg97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 19:31:36.512554 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512515 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.512554 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512552 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.512768 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512566 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttg97\" (UniqueName: \"kubernetes.io/projected/7e512eef-42aa-4f79-b483-c9f7886041ae-kube-api-access-ttg97\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.512768 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512580 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.512768 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512593 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e512eef-42aa-4f79-b483-c9f7886041ae-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.512768 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512605 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:36.512768 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:36.512616 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e512eef-42aa-4f79-b483-c9f7886041ae-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Jul 22 19:31:37.260922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:37.260885 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" event={"ID":"7e512eef-42aa-4f79-b483-c9f7886041ae","Type":"ContainerDied","Data":"05abb7717dd6e061abbb4646d4054f65ed49f46dc7860b539ab408bb4da02d70"} Jul 22 19:31:37.260922 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:37.260924 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f" Jul 22 19:31:37.261441 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:37.260947 2566 scope.go:117] "RemoveContainer" containerID="cd23afd043323d1706c50620413fd5058558781159ba41d457466579b5858410" Jul 22 19:31:37.283716 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:37.283686 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f"] Jul 22 19:31:37.287613 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:37.287588 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-8f2692-openshift-default-7679cd689f-n526f"] Jul 22 19:31:38.044983 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:38.044950 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" path="/var/lib/kubelet/pods/7e512eef-42aa-4f79-b483-c9f7886041ae/volumes" Jul 22 19:31:39.599634 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:39.599604 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-ftm8k_e1eb87b9-1554-49e5-a127-14de8f3ff8bf/copy-manifests/0.log" Jul 22 19:31:39.605840 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:39.605808 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-ftm8k_e1eb87b9-1554-49e5-a127-14de8f3ff8bf/manager/0.log" Jul 22 19:31:40.376442 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:40.376411 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29745780-wjqh8_12b86eb9-93ac-4535-8f10-577fe804110e/cleanup/0.log" Jul 22 19:31:41.654985 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:41.654951 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv_6c686071-ac4d-4c73-a57e-75bdb39deccb/pull/0.log" Jul 22 19:31:41.660991 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:41.660970 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv_6c686071-ac4d-4c73-a57e-75bdb39deccb/extract/0.log" Jul 22 19:31:41.667050 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:41.667033 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv_6c686071-ac4d-4c73-a57e-75bdb39deccb/util/0.log" Jul 22 19:31:41.794253 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:41.794224 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v_44643ad6-beea-4942-b96c-231e76efceb2/util/0.log" Jul 22 19:31:41.813378 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:41.813354 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v_44643ad6-beea-4942-b96c-231e76efceb2/pull/0.log" Jul 22 19:31:41.835833 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:41.835799 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v_44643ad6-beea-4942-b96c-231e76efceb2/extract/0.log" Jul 22 19:31:42.049861 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:42.049761 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg_36265b20-f4be-404a-a4dc-2d4bf169487d/pull/0.log" Jul 22 19:31:42.072510 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:42.072486 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg_36265b20-f4be-404a-a4dc-2d4bf169487d/extract/0.log" Jul 22 19:31:42.095883 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:42.095852 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg_36265b20-f4be-404a-a4dc-2d4bf169487d/util/0.log" Jul 22 19:31:42.220444 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:42.220413 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg_0e1d92ec-4681-4721-81f1-b9e100d16220/util/0.log" Jul 22 19:31:42.226294 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:42.226273 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg_0e1d92ec-4681-4721-81f1-b9e100d16220/pull/0.log" Jul 22 19:31:42.231761 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:42.231732 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg_0e1d92ec-4681-4721-81f1-b9e100d16220/extract/0.log" Jul 22 19:31:43.115442 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:43.115410 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-54bq2_8b15d19d-74e8-427c-9802-3c4b252f32dd/limitador/0.log" Jul 22 19:31:43.762894 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:43.762864 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8pwl_7f630aa9-42ed-40ad-befb-a325058d93be/discovery/0.log" Jul 22 19:31:44.641313 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:44.641282 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b8f49c9b6-w6xm6_93d054bc-ebb9-4490-9fbc-cfe880af248f/router/0.log" Jul 22 19:31:45.256519 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:45.256488 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8zhhc_baa82d72-e50f-476f-b8d8-9a8ce1cd6976/main/0.log" Jul 22 19:31:55.253798 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:55.253767 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8qw2t_5a250b6a-0e9c-4276-9d8e-3ccf85c7dd1a/global-pull-secret-syncer/0.log" Jul 22 19:31:55.429119 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:55.429086 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bt8k8_21492e71-6527-4603-bf9c-fec1c205cc81/konnectivity-agent/0.log" Jul 22 19:31:55.489805 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:55.489771 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-148.ec2.internal_0069f85ef90aeb8dad9e048240ec9720/haproxy/0.log" Jul 22 19:31:59.466237 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.466204 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv_6c686071-ac4d-4c73-a57e-75bdb39deccb/extract/0.log" Jul 22 19:31:59.490116 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.490087 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv_6c686071-ac4d-4c73-a57e-75bdb39deccb/util/0.log" Jul 22 19:31:59.513826 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.513800 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7596pjfv_6c686071-ac4d-4c73-a57e-75bdb39deccb/pull/0.log" Jul 22 19:31:59.625186 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.625159 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v_44643ad6-beea-4942-b96c-231e76efceb2/extract/0.log" Jul 22 19:31:59.645006 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.644978 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v_44643ad6-beea-4942-b96c-231e76efceb2/util/0.log" Jul 22 19:31:59.667110 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.667080 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e09m94v_44643ad6-beea-4942-b96c-231e76efceb2/pull/0.log" Jul 22 19:31:59.708730 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.708659 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg_36265b20-f4be-404a-a4dc-2d4bf169487d/extract/0.log" Jul 22 19:31:59.729081 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.729056 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg_36265b20-f4be-404a-a4dc-2d4bf169487d/util/0.log" Jul 22 19:31:59.753206 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.753185 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735lqxg_36265b20-f4be-404a-a4dc-2d4bf169487d/pull/0.log" Jul 22 19:31:59.786411 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.786376 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg_0e1d92ec-4681-4721-81f1-b9e100d16220/extract/0.log" Jul 22 19:31:59.814448 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.814420 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg_0e1d92ec-4681-4721-81f1-b9e100d16220/util/0.log" Jul 22 19:31:59.836711 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:31:59.836685 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17rksg_0e1d92ec-4681-4721-81f1-b9e100d16220/pull/0.log" Jul 22 19:32:00.850431 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:00.850403 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-54bq2_8b15d19d-74e8-427c-9802-3c4b252f32dd/limitador/0.log" Jul 22 19:32:02.347434 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.347408 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/alertmanager/0.log" Jul 22 19:32:02.366765 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.366735 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/config-reloader/0.log" Jul 22 19:32:02.388960 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.388937 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/kube-rbac-proxy-web/0.log" Jul 22 19:32:02.409644 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.409625 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/kube-rbac-proxy/0.log" Jul 22 19:32:02.431691 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.431664 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/kube-rbac-proxy-metric/0.log" Jul 22 19:32:02.453447 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.453425 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/prom-label-proxy/0.log" Jul 22 19:32:02.473994 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.473975 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_fc467fd1-fc00-489d-bc49-85e800af651c/init-config-reloader/0.log" Jul 22 19:32:02.511974 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.511952 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-vwsv6_ff741955-344e-4739-98a4-91530fe2f3cf/cluster-monitoring-operator/0.log" Jul 22 19:32:02.598961 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.598883 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6c4657d6ff-dlc9q_450400fd-9b4e-4737-9b00-40de387bd711/metrics-server/0.log" Jul 22 19:32:02.626327 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.626302 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-k4hfb_2fd46e12-ba52-4558-961c-6e97a82496ec/monitoring-plugin/0.log" Jul 22 19:32:02.729273 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.729248 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rsctn_35e30752-2472-4f6e-b4a0-111a17f5a1a5/node-exporter/0.log" Jul 22 19:32:02.760323 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.760299 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rsctn_35e30752-2472-4f6e-b4a0-111a17f5a1a5/kube-rbac-proxy/0.log" Jul 22 19:32:02.780288 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:02.780264 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rsctn_35e30752-2472-4f6e-b4a0-111a17f5a1a5/init-textfile/0.log" Jul 22 19:32:03.240492 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.240407 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7877b9fcd5-4ptmf_5d6b1ef2-cacb-40f5-917d-814193a13f11/telemeter-client/0.log" Jul 22 19:32:03.259399 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.259378 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7877b9fcd5-4ptmf_5d6b1ef2-cacb-40f5-917d-814193a13f11/reload/0.log" Jul 22 19:32:03.286858 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.286835 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7877b9fcd5-4ptmf_5d6b1ef2-cacb-40f5-917d-814193a13f11/kube-rbac-proxy/0.log" Jul 22 19:32:03.317717 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.317689 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9888b94c-cm6sb_595ee520-ff95-4c36-998c-ddd7099c4049/thanos-query/0.log" Jul 22 19:32:03.338299 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.338268 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9888b94c-cm6sb_595ee520-ff95-4c36-998c-ddd7099c4049/kube-rbac-proxy-web/0.log" Jul 22 19:32:03.359526 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.359505 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9888b94c-cm6sb_595ee520-ff95-4c36-998c-ddd7099c4049/kube-rbac-proxy/0.log" Jul 22 19:32:03.380008 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.379985 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9888b94c-cm6sb_595ee520-ff95-4c36-998c-ddd7099c4049/prom-label-proxy/0.log" Jul 22 19:32:03.398337 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.398310 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9888b94c-cm6sb_595ee520-ff95-4c36-998c-ddd7099c4049/kube-rbac-proxy-rules/0.log" Jul 22 19:32:03.420336 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:03.420312 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d9888b94c-cm6sb_595ee520-ff95-4c36-998c-ddd7099c4049/kube-rbac-proxy-metrics/0.log" Jul 22 19:32:05.304848 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:05.304804 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/1.log" Jul 22 19:32:05.310180 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:05.310158 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-x5kfm_844ec409-9225-487f-a44e-c8b6d7db9682/console-operator/2.log" Jul 22 19:32:05.800436 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:05.800406 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-557bb65c57-kc7r7_4d65bb9c-0bcb-45f9-af67-114b623a7b49/console/0.log" Jul 22 19:32:05.830501 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:05.830471 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-tx8r5_9460965c-3154-4b38-bfd3-dfee19bd240d/download-server/0.log" Jul 22 19:32:06.326913 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.326875 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n"] Jul 22 19:32:06.327301 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327226 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" Jul 22 19:32:06.327301 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327250 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" Jul 22 19:32:06.327301 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327274 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6953172d-3592-40ae-b5d7-8180cc48567b" containerName="test-curl-7789-140041791079088" Jul 22 19:32:06.327301 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327283 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6953172d-3592-40ae-b5d7-8180cc48567b" containerName="test-curl-7789-140041791079088" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327316 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327324 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327337 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327345 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327440 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e512eef-42aa-4f79-b483-c9f7886041ae" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327455 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327464 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="341f6a1c-dde7-4165-9f2c-e83f3066f344" containerName="istio-proxy" Jul 22 19:32:06.327527 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.327480 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6953172d-3592-40ae-b5d7-8180cc48567b" containerName="test-curl-7789-140041791079088" Jul 22 19:32:06.329534 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.329517 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.336955 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.336927 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nfc7\"/\"kube-root-ca.crt\"" Jul 22 19:32:06.337152 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.336950 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4nfc7\"/\"openshift-service-ca.crt\"" Jul 22 19:32:06.337774 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.337751 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4nfc7\"/\"default-dockercfg-dxrwg\"" Jul 22 19:32:06.341261 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.341232 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n"] Jul 22 19:32:06.493288 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.493248 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-lib-modules\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.493481 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.493319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw5wt\" (UniqueName: \"kubernetes.io/projected/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-kube-api-access-gw5wt\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.493481 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.493359 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-podres\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.493481 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.493401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-sys\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.493481 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.493464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-proc\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.561763 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.561733 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-k5jtt_e35897d3-8bde-4bd0-9e76-efbd0b6fd71e/volume-data-source-validator/0.log" Jul 22 19:32:06.594425 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-proc\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594425 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-lib-modules\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594609 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw5wt\" (UniqueName: \"kubernetes.io/projected/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-kube-api-access-gw5wt\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594609 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594471 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-proc\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594609 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-podres\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594609 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594526 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-sys\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594609 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594595 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-lib-modules\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594808 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594626 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-podres\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.594808 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.594656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-sys\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.602017 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.601996 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw5wt\" (UniqueName: \"kubernetes.io/projected/a2f46f04-4946-4caa-b1a7-3b3d7dae46b4-kube-api-access-gw5wt\") pod \"perf-node-gather-daemonset-kc54n\" (UID: \"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4\") " pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.647195 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.647157 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:06.777755 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:06.777721 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n"] Jul 22 19:32:06.779560 ip-10-0-137-148 kubenswrapper[2566]: W0722 19:32:06.779533 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2f46f04_4946_4caa_b1a7_3b3d7dae46b4.slice/crio-3f2ee71bce2ce8cd858dea3a7a7541723c6454df14ec026dd98aa6ea3b8731a1 WatchSource:0}: Error finding container 3f2ee71bce2ce8cd858dea3a7a7541723c6454df14ec026dd98aa6ea3b8731a1: Status 404 returned error can't find the container with id 3f2ee71bce2ce8cd858dea3a7a7541723c6454df14ec026dd98aa6ea3b8731a1 Jul 22 19:32:07.423747 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.423711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" event={"ID":"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4","Type":"ContainerStarted","Data":"734fe810cc67c45ba498dfd12a53ebacdc62ed1b653a1713e9ed45bc3a54aa76"} Jul 22 19:32:07.424162 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.423855 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:07.424162 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.423876 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" event={"ID":"a2f46f04-4946-4caa-b1a7-3b3d7dae46b4","Type":"ContainerStarted","Data":"3f2ee71bce2ce8cd858dea3a7a7541723c6454df14ec026dd98aa6ea3b8731a1"} Jul 22 19:32:07.440594 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.440548 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" podStartSLOduration=1.440533098 podStartE2EDuration="1.440533098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 19:32:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 19:32:07.438165215 +0000 UTC m=+3135.955976130" watchObservedRunningTime="2026-07-22 19:32:07.440533098 +0000 UTC m=+3135.958344013" Jul 22 19:32:07.636274 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.636243 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ffc88_965c0244-14f9-4460-9e1a-62c696692420/dns/0.log" Jul 22 19:32:07.656171 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.656126 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ffc88_965c0244-14f9-4460-9e1a-62c696692420/kube-rbac-proxy/0.log" Jul 22 19:32:07.785733 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:07.785655 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bwrkx_422501a3-4f95-45c0-b601-33d2e4410917/dns-node-resolver/0.log" Jul 22 19:32:08.367110 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:08.367079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8mls5_65c4db89-8088-48e2-9bc3-ab21f0c74fef/node-ca/0.log" Jul 22 19:32:09.410633 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:09.410601 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8pwl_7f630aa9-42ed-40ad-befb-a325058d93be/discovery/0.log" Jul 22 19:32:09.639355 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:09.639329 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b8f49c9b6-w6xm6_93d054bc-ebb9-4490-9fbc-cfe880af248f/router/0.log" Jul 22 19:32:10.351383 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:10.351350 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tdkfp_71b46f3d-ad01-4915-91d3-03d2e769d731/serve-healthcheck-canary/0.log" Jul 22 19:32:10.789732 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:10.789700 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-skfzm_fbde1910-202e-40d4-87cc-fe1c3b781290/insights-operator/0.log" Jul 22 19:32:10.790244 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:10.790228 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-skfzm_fbde1910-202e-40d4-87cc-fe1c3b781290/insights-operator/1.log" Jul 22 19:32:10.933431 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:10.933401 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mhpbl_3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c/kube-rbac-proxy/0.log" Jul 22 19:32:10.951638 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:10.951615 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mhpbl_3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c/exporter/0.log" Jul 22 19:32:10.973330 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:10.973305 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mhpbl_3cfd6db7-5394-4b19-b9d9-8ecc9ce07c2c/extractor/0.log" Jul 22 19:32:13.439573 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:13.439543 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4nfc7/perf-node-gather-daemonset-kc54n" Jul 22 19:32:13.707897 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:13.707775 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-ftm8k_e1eb87b9-1554-49e5-a127-14de8f3ff8bf/manager/0.log" Jul 22 19:32:13.726381 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:13.726359 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f9bb58876-ftm8k_e1eb87b9-1554-49e5-a127-14de8f3ff8bf/copy-manifests/0.log" Jul 22 19:32:15.181411 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:15.181306 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b444cc6d8-jw2c7_7ccc85ee-e426-4f50-a57e-e36a5c9bd833/manager/0.log" Jul 22 19:32:15.684247 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:15.684215 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8pwl_7f630aa9-42ed-40ad-befb-a325058d93be/discovery/0.log" Jul 22 19:32:15.867919 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:15.867892 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b8f49c9b6-w6xm6_93d054bc-ebb9-4490-9fbc-cfe880af248f/router/0.log" Jul 22 19:32:17.157804 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:17.157764 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p8pwl_7f630aa9-42ed-40ad-befb-a325058d93be/discovery/0.log" Jul 22 19:32:17.351442 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:17.351409 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b8f49c9b6-w6xm6_93d054bc-ebb9-4490-9fbc-cfe880af248f/router/0.log" Jul 22 19:32:20.671625 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:20.671575 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-kcvp9_1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63/kube-storage-version-migrator-operator/1.log" Jul 22 19:32:20.672514 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:20.672492 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-kcvp9_1cf0d3c9-3ea6-4b3f-b8fd-ca5bd6e77d63/kube-storage-version-migrator-operator/0.log" Jul 22 19:32:21.881321 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.881288 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/kube-multus-additional-cni-plugins/0.log" Jul 22 19:32:21.902390 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.902361 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/egress-router-binary-copy/0.log" Jul 22 19:32:21.921499 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.921471 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/cni-plugins/0.log" Jul 22 19:32:21.940274 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.940246 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/bond-cni-plugin/0.log" Jul 22 19:32:21.958241 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.958219 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/routeoverride-cni/0.log" Jul 22 19:32:21.977036 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.977008 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/whereabouts-cni-bincopy/0.log" Jul 22 19:32:21.995453 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:21.995426 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vrfdj_ca1eed09-b1dc-4e7e-a5ec-52531ad3f7f2/whereabouts-cni/0.log" Jul 22 19:32:22.027619 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:22.027588 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fs7tf_bb989be8-2cc6-4c03-9331-710903d993a9/kube-multus/0.log" Jul 22 19:32:22.167014 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:22.166910 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sjfmj_cf81b03f-0e6d-48dc-bafa-95e60c0c50a9/network-metrics-daemon/0.log" Jul 22 19:32:22.184131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:22.184103 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sjfmj_cf81b03f-0e6d-48dc-bafa-95e60c0c50a9/kube-rbac-proxy/0.log" Jul 22 19:32:23.003411 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.003379 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-controller/0.log" Jul 22 19:32:23.020172 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.020110 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/0.log" Jul 22 19:32:23.020172 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.020144 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovn-acl-logging/1.log" Jul 22 19:32:23.041183 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.041140 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/kube-rbac-proxy-node/0.log" Jul 22 19:32:23.062258 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.062208 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/kube-rbac-proxy-ovn-metrics/0.log" Jul 22 19:32:23.077297 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.077256 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/northd/0.log" Jul 22 19:32:23.097131 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.097087 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/nbdb/0.log" Jul 22 19:32:23.116933 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.116900 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/sbdb/0.log" Jul 22 19:32:23.174785 ip-10-0-137-148 kubenswrapper[2566]: I0722 19:32:23.174752 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xm2d_5b4f7771-df8b-4870-a865-260e76638f68/ovnkube-controller/0.log"