Jul 18 01:48:13.176690 ip-10-0-137-56 systemd[1]: Starting Kubernetes Kubelet... Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.579823 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585176 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585185 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585189 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585193 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585196 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585199 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585201 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:13.600857 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585205 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585208 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585211 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585214 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585217 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585219 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585222 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585225 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585229 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585240 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585243 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585246 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585249 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585252 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585255 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585258 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585261 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585264 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585267 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585269 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:13.617153 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585271 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585274 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585276 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585279 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585282 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585284 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585287 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585289 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585291 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585294 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585296 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585298 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585301 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585304 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585307 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585309 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585312 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585315 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585317 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:13.619084 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585320 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585322 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585325 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585327 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585331 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585334 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585336 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585339 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585341 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585343 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585346 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585349 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585351 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585354 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585357 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585359 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585362 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585364 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585367 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585369 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:13.620692 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585372 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585374 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585377 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585379 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585399 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585402 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585406 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585410 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585413 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585415 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585419 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585421 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585424 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585426 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585429 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585432 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585434 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585437 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585439 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:13.623038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585442 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585445 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585447 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585450 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585454 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585457 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585460 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585464 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585466 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585469 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585471 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585474 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585476 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.585479 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586148 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586154 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586157 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586160 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586162 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:13.624962 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586167 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586171 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586174 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586177 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586179 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586182 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586184 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586187 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586190 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586193 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586196 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586198 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586201 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586203 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586206 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586208 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586211 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586213 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586217 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:13.626907 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586222 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586226 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586230 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586233 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586236 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586239 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586241 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586244 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586247 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586249 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586252 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586254 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586257 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586259 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586263 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586266 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586269 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586271 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586274 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:13.627535 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586277 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586279 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586282 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586284 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586287 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586289 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586292 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586294 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586297 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586300 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586303 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586305 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586309 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586311 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586314 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586317 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586319 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586322 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586324 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586327 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:13.629188 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586329 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586331 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586334 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586336 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586339 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586341 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586344 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586346 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586349 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586352 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586354 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586357 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586360 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586362 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586365 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586367 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586370 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586372 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586375 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586377 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:13.630716 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586380 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586399 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586401 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586404 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586406 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586409 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586412 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586414 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586416 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586419 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586422 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586424 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586427 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586430 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586433 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586435 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586825 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586831 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586834 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586837 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:13.631884 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586841 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586844 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586847 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586850 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586852 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586855 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586858 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586860 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586863 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586866 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586869 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586872 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586874 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586877 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586879 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586882 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586884 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586887 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586889 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586892 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:13.632936 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586895 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586897 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586900 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586902 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586904 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586907 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586910 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586912 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586915 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586917 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586920 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586922 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586925 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586928 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586930 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586937 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586939 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586942 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586944 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586947 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:13.633816 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586949 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586952 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586954 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586957 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586960 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586962 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586965 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586967 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586972 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586975 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586978 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586982 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586985 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586988 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586990 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586993 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586995 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.586999 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587001 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:13.634691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587004 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587006 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587009 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587011 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587014 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587016 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587019 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587021 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587025 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587029 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587033 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587035 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587038 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587040 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587043 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587045 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587047 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587050 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587052 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587055 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:13.635562 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587057 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587060 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587062 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587066 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587071 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587073 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587076 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587079 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587082 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587085 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587087 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587090 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587092 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587095 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587097 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587100 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587166 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587173 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587181 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587187 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 01:48:13.636633 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587197 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587203 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587209 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587213 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587216 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587219 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587222 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587225 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587228 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587231 2580 flags.go:64] FLAG: --cgroup-root="" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587234 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587237 2580 flags.go:64] FLAG: --client-ca-file="" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587240 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587242 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587246 2580 flags.go:64] FLAG: --cluster-domain="" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587249 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587254 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587257 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587260 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587263 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587266 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587269 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587272 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587275 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 18 01:48:13.637495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587278 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587280 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587284 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587286 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587290 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587293 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587296 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587299 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587302 2580 flags.go:64] FLAG: --enable-server="true" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587306 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587309 2580 flags.go:64] FLAG: --event-burst="100" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587312 2580 flags.go:64] FLAG: --event-qps="50" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587315 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587318 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587321 2580 flags.go:64] FLAG: --eviction-hard="" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587324 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587327 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587330 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587333 2580 flags.go:64] FLAG: --eviction-soft="" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587336 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587338 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587341 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587344 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587347 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587350 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 01:48:13.638781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587353 2580 flags.go:64] FLAG: --feature-gates="" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587356 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587359 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587362 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587365 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587368 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587371 2580 flags.go:64] FLAG: --help="false" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587374 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-137-56.ec2.internal" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587376 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587379 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587394 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587398 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587401 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587404 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587406 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587410 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587413 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587417 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587420 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587423 2580 flags.go:64] FLAG: --kube-reserved="" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587426 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587428 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587431 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587434 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 01:48:13.639882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587437 2580 flags.go:64] FLAG: --lock-file="" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587439 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587442 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587445 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587450 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587453 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587456 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587460 2580 flags.go:64] FLAG: --logging-format="text" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587463 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587466 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587469 2580 flags.go:64] FLAG: --manifest-url="" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587472 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587476 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587478 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587482 2580 flags.go:64] FLAG: --max-pods="110" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587485 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587488 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587491 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587494 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587496 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587499 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587502 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587509 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587514 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587517 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 01:48:13.640969 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587520 2580 flags.go:64] FLAG: --pod-cidr="" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587523 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587528 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587531 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587534 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587537 2580 flags.go:64] FLAG: --port="10250" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587540 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587542 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a37ac659a429dbfc" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587546 2580 flags.go:64] FLAG: --qos-reserved="" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587548 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587551 2580 flags.go:64] FLAG: --register-node="true" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587554 2580 flags.go:64] FLAG: --register-with-taints="" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587558 2580 flags.go:64] FLAG: --registry-burst="10" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587560 2580 flags.go:64] FLAG: --registry-qps="5" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587564 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587567 2580 flags.go:64] FLAG: --reserved-memory="" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587571 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587574 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587577 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587579 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587582 2580 flags.go:64] FLAG: --runonce="false" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587585 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587588 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587591 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587594 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 01:48:13.642062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587596 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587599 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587602 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587605 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587608 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587611 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587615 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587618 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587621 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587624 2580 flags.go:64] FLAG: --system-cgroups="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587627 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587632 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587635 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587638 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587646 2580 flags.go:64] FLAG: --tls-min-version="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587649 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587651 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587654 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587657 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587660 2580 flags.go:64] FLAG: --v="2" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587663 2580 flags.go:64] FLAG: --version="false" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587668 2580 flags.go:64] FLAG: --vmodule="" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587672 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.587675 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 01:48:13.645041 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587781 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587785 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587788 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587791 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587794 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587796 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587799 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587801 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587804 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587807 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587810 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587812 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587815 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587818 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587821 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587824 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587826 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587830 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587834 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587837 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:13.646140 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587840 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587843 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587846 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587848 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587851 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587853 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587856 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587858 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587861 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587864 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587867 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587869 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587872 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587874 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587877 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587879 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587882 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587884 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587886 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587889 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:13.647671 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587891 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587894 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587896 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587899 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587901 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587904 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587907 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587910 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587912 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587915 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587917 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587920 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587923 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587926 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587928 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587930 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587933 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587936 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587938 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:13.648564 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587941 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587943 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587947 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587949 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587952 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587954 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587957 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587960 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587962 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587965 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587967 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587969 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587972 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587975 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587977 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587980 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587982 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587986 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587990 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587994 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:13.649833 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587997 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.587999 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588002 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588005 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588008 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588012 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588014 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588018 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588022 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588025 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588028 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588030 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588033 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588036 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588039 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588042 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588045 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588047 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588050 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:13.650718 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.588053 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.588850 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.595643 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.595661 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.595683 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.595692 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595763 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595769 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595774 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595778 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595781 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595785 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595789 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595793 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595797 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:13.655150 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595801 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595805 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595810 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595814 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595818 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595821 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595825 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595828 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595836 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595840 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595844 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595849 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595852 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595857 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595861 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595865 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595869 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595873 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595876 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595880 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:13.656021 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595885 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595889 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595892 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595896 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595900 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595904 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595908 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595912 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595916 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595920 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595924 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595927 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595931 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595935 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595939 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595943 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595946 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595950 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595954 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595958 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:13.774299 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595963 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:13.692463 ip-10-0-137-56 systemd[1]: Started Kubernetes Kubelet. Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595967 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595971 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595974 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595978 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595982 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595988 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595993 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.595997 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596001 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596006 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596010 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596013 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596018 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596022 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596026 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596046 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596050 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596055 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:13.775038 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596060 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596064 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596068 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596072 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596076 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596080 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596084 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596088 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596092 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596095 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596099 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596103 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596107 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596112 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596117 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596122 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596126 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596131 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596135 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596139 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:13.775786 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596142 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596146 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596149 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596153 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596157 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596161 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596164 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596169 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596175 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596179 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596184 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.596190 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596341 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:13.776883 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596348 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596351 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596356 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596361 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596365 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596371 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596375 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596379 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596399 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596403 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596406 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596410 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596414 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596421 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596427 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596431 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596435 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596440 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596445 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:13.777333 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596449 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596453 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596457 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596460 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596464 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596468 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596472 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596475 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596480 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596483 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596487 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596491 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596494 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596498 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596502 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596505 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596509 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596513 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596518 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596522 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:13.777970 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596526 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596530 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596534 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596537 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596540 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596544 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596549 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596553 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596556 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596560 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596564 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596567 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596571 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596575 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596578 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596582 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596586 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596589 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596592 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596596 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:13.778655 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596600 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596604 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596607 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596611 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596614 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596618 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596622 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596625 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596629 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596633 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596637 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596641 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596645 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596649 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596652 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596656 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596660 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596663 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596669 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:13.779277 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596673 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596677 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596680 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596684 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596688 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596691 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596695 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596698 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596702 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596706 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596709 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596713 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596716 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596720 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596724 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596728 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596732 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596735 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596739 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:13.779846 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:13.596743 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.596749 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.596867 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.600680 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.602185 2580 server.go:1019] "Starting client certificate rotation" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.602286 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.602327 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.605784 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.608971 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.625166 2580 log.go:25] "Validated CRI v1 runtime API" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.627274 2580 log.go:25] "Validated CRI v1 image API" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.628966 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 01:48:13.780565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.660737 2580 fs.go:135] Filesystem UUIDs: map[48775e22-778e-48e3-928a-1d7aabdbc648:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 962b3da7-e136-46b4-adbd-b57fcc6f78c5:/dev/nvme0n1p3] Jul 18 01:48:13.780976 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.660753 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 01:48:13.780976 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.664891 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:48:13.780976 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.665040 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdd9k" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.668423 2580 manager.go:216] Machine: {Timestamp:2026-07-18 01:48:13.667121355 +0000 UTC m=+0.361190996 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090947 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b074e26d73f8b49cf14421b0ec848 SystemUUID:ec2b074e-26d7-3f8b-49cf-14421b0ec848 BootID:5164f5ab-916c-42bb-afb8-2bf637557cec Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a2:ef:0e:6a:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a2:ef:0e:6a:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:87:45:23:cf:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.668591 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.668711 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.671929 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.671953 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-56.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.672623 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.672633 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.672640 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.674031 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.675886 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679425 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679532 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679561 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679575 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679588 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679596 2580 kubelet.go:398] "Adding apiserver pod source" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679605 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 01:48:13.781310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679848 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.679861 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.684139 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.686595 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.686623 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687876 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687889 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687895 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687900 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687908 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687913 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687921 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687926 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687932 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687937 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687947 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.687952 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.688780 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.688788 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.691673 2580 server.go:1268] "Started kubelet" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.691751 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.691746 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.691810 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.692957 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.693417 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.694647 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.696349 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.697536 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.697950 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-56.ec2.internal" not found Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.706688 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.706771 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.706792 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-56.ec2.internal\" not found" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707343 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707353 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707406 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707493 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707500 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707598 2580 factory.go:55] Registering systemd factory Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707614 2580 factory.go:223] Registration of the systemd container factory successfully Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707828 2580 factory.go:153] Registering CRI-O factory Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707837 2580 factory.go:223] Registration of the crio container factory successfully Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707876 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707898 2580 factory.go:103] Registering Raw factory Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.707910 2580 manager.go:1196] Started watching for new ooms in manager Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.708265 2580 manager.go:349] Starting recovery of all containers Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.708796 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.718899 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-56.ec2.internal\" not found" node="ip-10-0-137-56.ec2.internal" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.720969 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-56.ec2.internal" not found Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.727464 2580 manager.go:354] Recovery completed Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.734060 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.735617 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:13.944910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.735643 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.735654 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736087 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736096 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736099 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736111 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736786 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736807 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.736816 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.737217 2580 policy_none.go:49] "None policy: Start" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.737229 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.737239 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.738225 2580 policy_none.go:47] "Start" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.776691 2580 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-56.ec2.internal" not found Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792339 2580 manager.go:337] "Starting Device Plugin manager" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.792369 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792378 2580 server.go:87] "Starting device plugin registration server" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792601 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792613 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792715 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792799 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.792808 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.793159 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.793210 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-56.ec2.internal\" not found" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.879497 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.880596 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.880635 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.880657 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.880689 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.882969 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.893099 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.893776 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.893801 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.893812 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.893831 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-56.ec2.internal" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.902974 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-56.ec2.internal" Jul 18 01:48:13.961554 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:13.902994 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-56.ec2.internal\": node \"ip-10-0-137-56.ec2.internal\" not found" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.981077 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal"] Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.982364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:13.983250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.019167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b6887dcbefc1c78a82854291c613ebc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal\" (UID: \"2b6887dcbefc1c78a82854291c613ebc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.019319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b6887dcbefc1c78a82854291c613ebc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal\" (UID: \"2b6887dcbefc1c78a82854291c613ebc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.019335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/14307c8ae8c30716921eb918c5798d1d-config\") pod \"kube-apiserver-proxy-ip-10-0-137-56.ec2.internal\" (UID: \"14307c8ae8c30716921eb918c5798d1d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.026365 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.030678 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.036899 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.042531 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.119608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b6887dcbefc1c78a82854291c613ebc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal\" (UID: \"2b6887dcbefc1c78a82854291c613ebc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.119628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b6887dcbefc1c78a82854291c613ebc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal\" (UID: \"2b6887dcbefc1c78a82854291c613ebc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.119644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/14307c8ae8c30716921eb918c5798d1d-config\") pod \"kube-apiserver-proxy-ip-10-0-137-56.ec2.internal\" (UID: \"14307c8ae8c30716921eb918c5798d1d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.119684 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/14307c8ae8c30716921eb918c5798d1d-config\") pod \"kube-apiserver-proxy-ip-10-0-137-56.ec2.internal\" (UID: \"14307c8ae8c30716921eb918c5798d1d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.119710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b6887dcbefc1c78a82854291c613ebc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal\" (UID: \"2b6887dcbefc1c78a82854291c613ebc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.155882 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.119717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b6887dcbefc1c78a82854291c613ebc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal\" (UID: \"2b6887dcbefc1c78a82854291c613ebc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.342535 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.340544 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.347180 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.346683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.611902 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.612028 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.612062 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.612063 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.681753 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:43:14 +0000 UTC" deadline="2028-02-07 14:01:43.905319445 +0000 UTC" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.681788 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13668h13m28.223533151s" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.693891 2580 apiserver.go:52] "Watching apiserver" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.699129 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.699456 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-lmppk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal","openshift-multus/multus-additional-cni-plugins-dqb6d","openshift-multus/multus-qb2lp","openshift-network-diagnostics/network-check-target-frd7c","kube-system/konnectivity-agent-nt7fw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4","openshift-dns/node-resolver-8xsvd","openshift-image-registry/node-ca-87z6p","openshift-multus/network-metrics-daemon-4xj87","openshift-network-operator/iptables-alerter-966z9","openshift-ovn-kubernetes/ovnkube-node-kcwzg","kube-system/global-pull-secret-syncer-rf5pp","kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal"] Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.700932 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.703428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:15.703568 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.703476 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.703752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.704014 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.704298 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.704433 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.704477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dmbfz\"" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.705117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.706984 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.706122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.709973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.707262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:15.709973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.707348 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.709973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.707594 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fnm9l\"" Jul 18 01:48:15.709973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.708506 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.709973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.709416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.712540 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.710694 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:15.712540 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.710733 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:15.712540 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.712000 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.712540 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.712038 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:15.714463 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.713232 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.714463 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.713970 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.719560 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.718025 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722366 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-kubelet\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-kubelet\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722856 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-modprobe-d\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fbsc\" (UniqueName: \"kubernetes.io/projected/58882a96-d369-4022-88bb-7c6222deacd6-kube-api-access-2fbsc\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f3cf71-8c8d-40af-8004-0b7886594448-tmp-dir\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-multus-certs\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-ovnkube-config\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.722996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50f3cf71-8c8d-40af-8004-0b7886594448-hosts-file\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-cni-multus\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723053 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-hostroot\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-node-log\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723093 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-ovnkube-script-lib\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723107 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-sys\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9chqj\" (UniqueName: \"kubernetes.io/projected/19f50b49-b7d3-4df2-9619-4464c6bf21ba-kube-api-access-9chqj\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-system-cni-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.831183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9af9005-775c-4224-9567-909f4795d169-cni-binary-copy\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-conf-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-slash\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-systemd\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-cni-bin\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-ovn\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysctl-d\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-systemd\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-socket-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723374 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-dbus\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723409 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-cnibin\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-run-netns\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723465 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-log-socket\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/891e9aab-d18d-4130-94b5-e1ec8af5717b-serviceca\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb03f2c1-c046-416e-a6e6-85f56efe4964-host-slash\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.832265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-etc-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-system-cni-dir\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-cni-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723639 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-etc-kubernetes\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d945w\" (UniqueName: \"kubernetes.io/projected/891e9aab-d18d-4130-94b5-e1ec8af5717b-kube-api-access-d945w\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-kubelet-config\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723713 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-socket-dir-parent\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-var-lib-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723749 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723749 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-cni-netd\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/891e9aab-d18d-4130-94b5-e1ec8af5717b-host\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-registration-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-sys-fs\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqh9q\" (UniqueName: \"kubernetes.io/projected/fb03f2c1-c046-416e-a6e6-85f56efe4964-kube-api-access-tqh9q\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723916 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-run\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.835417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58882a96-d369-4022-88bb-7c6222deacd6-etc-tuned\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.723976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-cnibin\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-cni-binary-copy\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724018 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724057 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe-agent-certs\") pod \"konnectivity-agent-nt7fw\" (UID: \"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe\") " pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724128 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe-konnectivity-ca\") pod \"konnectivity-agent-nt7fw\" (UID: \"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe\") " pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/880e0b4b-9383-434f-8338-9834f1240186-ovn-node-metrics-cert\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-host\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-cni-bin\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crq67\" (UniqueName: \"kubernetes.io/projected/e9af9005-775c-4224-9567-909f4795d169-kube-api-access-crq67\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-k8s-cni-cncf-io\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2765\" (UniqueName: \"kubernetes.io/projected/880e0b4b-9383-434f-8338-9834f1240186-kube-api-access-p2765\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.836296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl272\" (UniqueName: \"kubernetes.io/projected/ca5dcb47-0098-492e-a219-40954bd79b4e-kube-api-access-rl272\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724311 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-run-ovn-kubernetes\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysconfig\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysctl-conf\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724368 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-lib-modules\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58882a96-d369-4022-88bb-7c6222deacd6-tmp\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-os-release\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97bz\" (UniqueName: \"kubernetes.io/projected/a79aa2c5-6ecd-4434-a381-adc4351be14a-kube-api-access-w97bz\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-kubernetes\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-device-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvpp\" (UniqueName: \"kubernetes.io/projected/50f3cf71-8c8d-40af-8004-0b7886594448-kube-api-access-llvpp\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb03f2c1-c046-416e-a6e6-85f56efe4964-iptables-alerter-script\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-os-release\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-netns\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.839999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9af9005-775c-4224-9567-909f4795d169-multus-daemon-config\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-systemd-units\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-env-overrides\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-var-lib-kubelet\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724787 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.724789 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.725084 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.725660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qffkh\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.725791 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.725846 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726018 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p7cgk\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726079 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h7j7q\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726128 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726193 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726293 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4d6w\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726884 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726922 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726932 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.726937 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.727166 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.727576 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.727721 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.727786 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.727848 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.727873 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.728027 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.728097 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-llt6v\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.728221 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pl4mr\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.728338 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 01:48:15.840808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.728390 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zxhpn\"" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.735245 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.776890 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nwd62" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.796835 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nwd62" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysctl-conf\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-lib-modules\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58882a96-d369-4022-88bb-7c6222deacd6-tmp\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825638 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-os-release\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97bz\" (UniqueName: \"kubernetes.io/projected/a79aa2c5-6ecd-4434-a381-adc4351be14a-kube-api-access-w97bz\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-kubernetes\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-device-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825710 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llvpp\" (UniqueName: \"kubernetes.io/projected/50f3cf71-8c8d-40af-8004-0b7886594448-kube-api-access-llvpp\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb03f2c1-c046-416e-a6e6-85f56efe4964-iptables-alerter-script\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825745 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-os-release\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-netns\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9af9005-775c-4224-9567-909f4795d169-multus-daemon-config\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825789 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-systemd-units\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.842322 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825815 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-env-overrides\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-var-lib-kubelet\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-kubelet\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-kubelet\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-modprobe-d\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fbsc\" (UniqueName: \"kubernetes.io/projected/58882a96-d369-4022-88bb-7c6222deacd6-kube-api-access-2fbsc\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f3cf71-8c8d-40af-8004-0b7886594448-tmp-dir\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-multus-certs\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-ovnkube-config\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50f3cf71-8c8d-40af-8004-0b7886594448-hosts-file\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.825991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-cni-multus\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-hostroot\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-node-log\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-ovnkube-script-lib\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826055 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-sys\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9chqj\" (UniqueName: \"kubernetes.io/projected/19f50b49-b7d3-4df2-9619-4464c6bf21ba-kube-api-access-9chqj\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-system-cni-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.845543 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9af9005-775c-4224-9567-909f4795d169-cni-binary-copy\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-conf-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-slash\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-systemd\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-cni-bin\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-ovn\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysctl-d\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-systemd\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826261 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-socket-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-dbus\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-cnibin\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-run-netns\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-log-socket\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/891e9aab-d18d-4130-94b5-e1ec8af5717b-serviceca\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb03f2c1-c046-416e-a6e6-85f56efe4964-host-slash\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-etc-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.846433 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-system-cni-dir\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-cni-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-etc-kubernetes\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d945w\" (UniqueName: \"kubernetes.io/projected/891e9aab-d18d-4130-94b5-e1ec8af5717b-kube-api-access-d945w\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-kubelet-config\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-socket-dir-parent\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-var-lib-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-cni-netd\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826504 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/891e9aab-d18d-4130-94b5-e1ec8af5717b-host\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-registration-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-sys-fs\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqh9q\" (UniqueName: \"kubernetes.io/projected/fb03f2c1-c046-416e-a6e6-85f56efe4964-kube-api-access-tqh9q\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-run\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58882a96-d369-4022-88bb-7c6222deacd6-etc-tuned\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-cnibin\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.847320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-cni-binary-copy\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe-agent-certs\") pod \"konnectivity-agent-nt7fw\" (UID: \"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe\") " pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe-konnectivity-ca\") pod \"konnectivity-agent-nt7fw\" (UID: \"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe\") " pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/880e0b4b-9383-434f-8338-9834f1240186-ovn-node-metrics-cert\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-host\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-cni-bin\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crq67\" (UniqueName: \"kubernetes.io/projected/e9af9005-775c-4224-9567-909f4795d169-kube-api-access-crq67\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826832 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-k8s-cni-cncf-io\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2765\" (UniqueName: \"kubernetes.io/projected/880e0b4b-9383-434f-8338-9834f1240186-kube-api-access-p2765\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl272\" (UniqueName: \"kubernetes.io/projected/ca5dcb47-0098-492e-a219-40954bd79b4e-kube-api-access-rl272\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-run-ovn-kubernetes\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysconfig\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.826999 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysconfig\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.849353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827020 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-kubelet\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827075 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-host\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827076 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-cnibin\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827088 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-socket-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827088 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysctl-d\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-systemd\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-device-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-modprobe-d\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-systemd-units\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827431 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827474 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-os-release\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827490 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-netns\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-cni-binary-copy\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827659 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-run\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827669 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.827765 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:15.850318 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.827834 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.327813196 +0000 UTC m=+2.011685932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:15.852914 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50f3cf71-8c8d-40af-8004-0b7886594448-tmp-dir\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.852914 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.827919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-multus-certs\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828130 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e9af9005-775c-4224-9567-909f4795d169-multus-daemon-config\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828161 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-env-overrides\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828209 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-var-lib-kubelet\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828279 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe-konnectivity-ca\") pod \"konnectivity-agent-nt7fw\" (UID: \"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe\") " pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828347 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-ovnkube-config\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828389 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/50f3cf71-8c8d-40af-8004-0b7886594448-hosts-file\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828416 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-cni-multus\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-hostroot\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828457 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-cni-bin\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-node-log\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828612 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-run-k8s-cni-cncf-io\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-slash\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828700 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-sys\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828767 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-system-cni-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828853 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-lib-modules\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.828868 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.863494 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.363478641 +0000 UTC m=+2.047351361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:15.919653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828927 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/880e0b4b-9383-434f-8338-9834f1240186-ovnkube-script-lib\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-sysctl-conf\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828977 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828980 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-os-release\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829005 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-conf-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829015 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-systemd\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829030 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-cni-bin\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-host-var-lib-kubelet\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829060 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-run-ovn\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829099 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829179 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9af9005-775c-4224-9567-909f4795d169-cni-binary-copy\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829180 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-dbus\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb03f2c1-c046-416e-a6e6-85f56efe4964-host-slash\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829228 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a79aa2c5-6ecd-4434-a381-adc4351be14a-system-cni-dir\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.828896 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-run-ovn-kubernetes\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-etc-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829257 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-run-netns\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.863632 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:15.996746 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.863653 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.863668 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829304 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-cni-dir\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829383 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-sys-fs\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829422 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-registration-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.863706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a79aa2c5-6ecd-4434-a381-adc4351be14a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829340 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-etc-kubernetes\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fb03f2c1-c046-416e-a6e6-85f56efe4964-iptables-alerter-script\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:15.863728 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.363711744 +0000 UTC m=+2.047584461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-multus-socket-dir-parent\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829535 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca5dcb47-0098-492e-a219-40954bd79b4e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58882a96-d369-4022-88bb-7c6222deacd6-etc-kubernetes\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.832128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/58882a96-d369-4022-88bb-7c6222deacd6-tmp\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829777 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/891e9aab-d18d-4130-94b5-e1ec8af5717b-serviceca\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.832247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/58882a96-d369-4022-88bb-7c6222deacd6-etc-tuned\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.832677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/880e0b4b-9383-434f-8338-9834f1240186-ovn-node-metrics-cert\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:15.999972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829290 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-kubelet-config\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829243 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9af9005-775c-4224-9567-909f4795d169-cnibin\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829280 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-log-socket\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829455 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-var-lib-openvswitch\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829308 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/891e9aab-d18d-4130-94b5-e1ec8af5717b-host\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.829532 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/880e0b4b-9383-434f-8338-9834f1240186-host-cni-netd\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.833297 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe-agent-certs\") pod \"konnectivity-agent-nt7fw\" (UID: \"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe\") " pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.865945 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqh9q\" (UniqueName: \"kubernetes.io/projected/fb03f2c1-c046-416e-a6e6-85f56efe4964-kube-api-access-tqh9q\") pod \"iptables-alerter-966z9\" (UID: \"fb03f2c1-c046-416e-a6e6-85f56efe4964\") " pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.866067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97bz\" (UniqueName: \"kubernetes.io/projected/a79aa2c5-6ecd-4434-a381-adc4351be14a-kube-api-access-w97bz\") pod \"multus-additional-cni-plugins-dqb6d\" (UID: \"a79aa2c5-6ecd-4434-a381-adc4351be14a\") " pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.866140 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl272\" (UniqueName: \"kubernetes.io/projected/ca5dcb47-0098-492e-a219-40954bd79b4e-kube-api-access-rl272\") pod \"aws-ebs-csi-driver-node-7bjt4\" (UID: \"ca5dcb47-0098-492e-a219-40954bd79b4e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.866166 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fbsc\" (UniqueName: \"kubernetes.io/projected/58882a96-d369-4022-88bb-7c6222deacd6-kube-api-access-2fbsc\") pod \"tuned-lmppk\" (UID: \"58882a96-d369-4022-88bb-7c6222deacd6\") " pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.866330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d945w\" (UniqueName: \"kubernetes.io/projected/891e9aab-d18d-4130-94b5-e1ec8af5717b-kube-api-access-d945w\") pod \"node-ca-87z6p\" (UID: \"891e9aab-d18d-4130-94b5-e1ec8af5717b\") " pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.866366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvpp\" (UniqueName: \"kubernetes.io/projected/50f3cf71-8c8d-40af-8004-0b7886594448-kube-api-access-llvpp\") pod \"node-resolver-8xsvd\" (UID: \"50f3cf71-8c8d-40af-8004-0b7886594448\") " pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.866401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9chqj\" (UniqueName: \"kubernetes.io/projected/19f50b49-b7d3-4df2-9619-4464c6bf21ba-kube-api-access-9chqj\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.871386 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crq67\" (UniqueName: \"kubernetes.io/projected/e9af9005-775c-4224-9567-909f4795d169-kube-api-access-crq67\") pod \"multus-qb2lp\" (UID: \"e9af9005-775c-4224-9567-909f4795d169\") " pod="openshift-multus/multus-qb2lp" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.881858 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2765\" (UniqueName: \"kubernetes.io/projected/880e0b4b-9383-434f-8338-9834f1240186-kube-api-access-p2765\") pod \"ovnkube-node-kcwzg\" (UID: \"880e0b4b-9383-434f-8338-9834f1240186\") " pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.886917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xsvd" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.893630 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.945336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:16.000855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.954727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-966z9" Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:15.971784 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb03f2c1_c046_416e_a6e6_85f56efe4964.slice/crio-957a35f6a097e13c2132686d5321f75e94a46b0ddf91e2432f80d0be827e1690 WatchSource:0}: Error finding container 957a35f6a097e13c2132686d5321f75e94a46b0ddf91e2432f80d0be827e1690: Status 404 returned error can't find the container with id 957a35f6a097e13c2132686d5321f75e94a46b0ddf91e2432f80d0be827e1690 Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:15.971986 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6887dcbefc1c78a82854291c613ebc.slice/crio-19024b0d31fe6af4f3bfad4fe90adb70e272954e005ddd31d721326ab012f372 WatchSource:0}: Error finding container 19024b0d31fe6af4f3bfad4fe90adb70e272954e005ddd31d721326ab012f372: Status 404 returned error can't find the container with id 19024b0d31fe6af4f3bfad4fe90adb70e272954e005ddd31d721326ab012f372 Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:15.972185 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14307c8ae8c30716921eb918c5798d1d.slice/crio-9c91f56d7eb8195cbecf87bd8d2b3643adeadc53ffa8cfa7c1c76c45c037a6ef WatchSource:0}: Error finding container 9c91f56d7eb8195cbecf87bd8d2b3643adeadc53ffa8cfa7c1c76c45c037a6ef: Status 404 returned error can't find the container with id 9c91f56d7eb8195cbecf87bd8d2b3643adeadc53ffa8cfa7c1c76c45c037a6ef Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:15.972384 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod880e0b4b_9383_434f_8338_9834f1240186.slice/crio-691ec834e7ce1f1b353def6c2028dd91ea248e783097b90772773ae2fe57c001 WatchSource:0}: Error finding container 691ec834e7ce1f1b353def6c2028dd91ea248e783097b90772773ae2fe57c001: Status 404 returned error can't find the container with id 691ec834e7ce1f1b353def6c2028dd91ea248e783097b90772773ae2fe57c001 Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.975413 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.975925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87z6p" Jul 18 01:48:16.002782 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:15.981870 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lmppk" Jul 18 01:48:16.067351 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.067329 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" Jul 18 01:48:16.074475 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:16.074455 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5dcb47_0098_492e_a219_40954bd79b4e.slice/crio-96c3790e1cf6e197dc45a63f820e11c6b65ccea7c8968e6175a1b0bf416e2e1d WatchSource:0}: Error finding container 96c3790e1cf6e197dc45a63f820e11c6b65ccea7c8968e6175a1b0bf416e2e1d: Status 404 returned error can't find the container with id 96c3790e1cf6e197dc45a63f820e11c6b65ccea7c8968e6175a1b0bf416e2e1d Jul 18 01:48:16.093014 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.092999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" Jul 18 01:48:16.098928 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:16.098906 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79aa2c5_6ecd_4434_a381_adc4351be14a.slice/crio-4713ea36bf2bd0864632ca3e4aa4945d5d2ed8eeac8ab91ed0c6310d46c41c67 WatchSource:0}: Error finding container 4713ea36bf2bd0864632ca3e4aa4945d5d2ed8eeac8ab91ed0c6310d46c41c67: Status 404 returned error can't find the container with id 4713ea36bf2bd0864632ca3e4aa4945d5d2ed8eeac8ab91ed0c6310d46c41c67 Jul 18 01:48:16.119138 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.119117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qb2lp" Jul 18 01:48:16.125042 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:16.125024 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9af9005_775c_4224_9567_909f4795d169.slice/crio-02807b6be0079b5decd059512dfad5f506fc4700446cf474d4a40848342a1806 WatchSource:0}: Error finding container 02807b6be0079b5decd059512dfad5f506fc4700446cf474d4a40848342a1806: Status 404 returned error can't find the container with id 02807b6be0079b5decd059512dfad5f506fc4700446cf474d4a40848342a1806 Jul 18 01:48:16.147881 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:16.147864 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f3cf71_8c8d_40af_8004_0b7886594448.slice/crio-b84e32d42425391d00a2d1f6f6d29e0cb468a5689740d7bf64a8424f689b75ce WatchSource:0}: Error finding container b84e32d42425391d00a2d1f6f6d29e0cb468a5689740d7bf64a8424f689b75ce: Status 404 returned error can't find the container with id b84e32d42425391d00a2d1f6f6d29e0cb468a5689740d7bf64a8424f689b75ce Jul 18 01:48:16.330253 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.330186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:16.330393 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.330343 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:16.330457 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.330412 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.330390894 +0000 UTC m=+3.014263624 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.430523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.430605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.430781 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.430843 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.430823792 +0000 UTC m=+3.114696510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.431244 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.431263 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.431276 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:16.431353 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.431321 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.431305794 +0000 UTC m=+3.115178515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:16.439944 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:16.439915 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236ccd4d_b5ea_4d2d_9acf_6d7daefeb8fe.slice/crio-eff53c22b6c1823897c4b9c34f31e623ebd294d4c0655d2f7d2669c68c8a1c27 WatchSource:0}: Error finding container eff53c22b6c1823897c4b9c34f31e623ebd294d4c0655d2f7d2669c68c8a1c27: Status 404 returned error can't find the container with id eff53c22b6c1823897c4b9c34f31e623ebd294d4c0655d2f7d2669c68c8a1c27 Jul 18 01:48:16.692349 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.691851 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:16.799320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.799214 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:15 +0000 UTC" deadline="2028-01-07 07:27:50.333872065 +0000 UTC" Jul 18 01:48:16.799320 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.799278 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12917h39m33.534598218s" Jul 18 01:48:16.822956 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.822928 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:16.891839 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.891803 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:16.891994 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:16.891923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:16.921170 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.921096 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" event={"ID":"2b6887dcbefc1c78a82854291c613ebc","Type":"ContainerStarted","Data":"19024b0d31fe6af4f3bfad4fe90adb70e272954e005ddd31d721326ab012f372"} Jul 18 01:48:16.931680 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.931650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerStarted","Data":"4713ea36bf2bd0864632ca3e4aa4945d5d2ed8eeac8ab91ed0c6310d46c41c67"} Jul 18 01:48:16.952572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.950596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lmppk" event={"ID":"58882a96-d369-4022-88bb-7c6222deacd6","Type":"ContainerStarted","Data":"1a0ad61d0a8783f217e0243195a7c13ad4dead5c988c61b2056f59825b90b254"} Jul 18 01:48:16.953712 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.953552 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87z6p" event={"ID":"891e9aab-d18d-4130-94b5-e1ec8af5717b","Type":"ContainerStarted","Data":"3ad729c93dac1a6adf3734c73d7f14592191d2233338dec001a8f49b1ec95ef2"} Jul 18 01:48:16.973191 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.973169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-966z9" event={"ID":"fb03f2c1-c046-416e-a6e6-85f56efe4964","Type":"ContainerStarted","Data":"957a35f6a097e13c2132686d5321f75e94a46b0ddf91e2432f80d0be827e1690"} Jul 18 01:48:16.980434 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.980377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"691ec834e7ce1f1b353def6c2028dd91ea248e783097b90772773ae2fe57c001"} Jul 18 01:48:16.993950 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:16.993926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" event={"ID":"14307c8ae8c30716921eb918c5798d1d","Type":"ContainerStarted","Data":"9c91f56d7eb8195cbecf87bd8d2b3643adeadc53ffa8cfa7c1c76c45c037a6ef"} Jul 18 01:48:17.008484 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.008409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nt7fw" event={"ID":"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe","Type":"ContainerStarted","Data":"eff53c22b6c1823897c4b9c34f31e623ebd294d4c0655d2f7d2669c68c8a1c27"} Jul 18 01:48:17.023521 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.023488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xsvd" event={"ID":"50f3cf71-8c8d-40af-8004-0b7886594448","Type":"ContainerStarted","Data":"b84e32d42425391d00a2d1f6f6d29e0cb468a5689740d7bf64a8424f689b75ce"} Jul 18 01:48:17.027927 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.027905 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qb2lp" event={"ID":"e9af9005-775c-4224-9567-909f4795d169","Type":"ContainerStarted","Data":"02807b6be0079b5decd059512dfad5f506fc4700446cf474d4a40848342a1806"} Jul 18 01:48:17.047525 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.047448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" event={"ID":"ca5dcb47-0098-492e-a219-40954bd79b4e","Type":"ContainerStarted","Data":"96c3790e1cf6e197dc45a63f820e11c6b65ccea7c8968e6175a1b0bf416e2e1d"} Jul 18 01:48:17.127405 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.127316 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:17.340142 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.339962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:17.340142 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.340138 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:17.340349 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.340195 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:48:19.340178329 +0000 UTC m=+5.024051060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:17.441162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.441130 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:17.441323 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.441194 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:17.441323 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.441298 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:17.441323 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.441304 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:17.441487 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.441328 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:17.441487 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.441340 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:17.441487 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.441361 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:48:19.441343294 +0000 UTC m=+5.125216016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:17.441487 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.441409 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:19.441390455 +0000 UTC m=+5.125263175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:17.799859 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.799724 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:15 +0000 UTC" deadline="2028-04-23 09:44:24.27565161 +0000 UTC" Jul 18 01:48:17.799859 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.799768 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15487h56m6.475889013s" Jul 18 01:48:17.891342 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.891305 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:17.891540 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.891439 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:17.891875 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:17.891853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:17.891990 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:17.891967 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:18.893434 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:18.893405 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:18.893912 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:18.893540 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:19.358706 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:19.358667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:19.358860 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.358807 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:19.358931 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.358869 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:48:23.358849731 +0000 UTC m=+9.042722465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:19.460198 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:19.459933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:19.460198 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:19.460000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:19.460198 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.460119 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:19.460198 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.460149 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:19.460198 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.460162 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:19.460548 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.460220 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:23.460199137 +0000 UTC m=+9.144071868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:19.460548 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.460124 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:19.460548 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.460269 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:48:23.460259245 +0000 UTC m=+9.144131967 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:19.892334 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:19.891814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:19.892334 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:19.891860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:19.892334 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.891947 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:19.892334 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:19.892078 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:20.892615 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:20.891621 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:20.892615 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:20.891749 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:21.891737 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:21.891703 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:21.891737 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:21.891728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:21.891978 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:21.891848 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:21.891978 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:21.891923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:22.891995 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:22.891952 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:22.892446 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:22.892094 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:23.392154 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:23.392085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:23.392331 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.392252 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:23.392331 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.392323 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:48:31.392303215 +0000 UTC m=+17.076175940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:23.493350 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:23.493310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:23.493508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:23.493377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:23.493508 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.493494 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:23.493664 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.493505 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:23.493664 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.493524 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:23.493664 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.493532 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:23.493664 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.493555 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:48:31.493535416 +0000 UTC m=+17.177408134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:23.493664 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.493603 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:31.493572852 +0000 UTC m=+17.177445591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:23.891258 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:23.891191 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:23.891378 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:23.891198 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:23.891378 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.891341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:23.891480 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:23.891388 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:24.892140 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:24.892084 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:24.892573 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:24.892204 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:25.891233 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:25.891153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:25.891390 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:25.891250 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:25.891390 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:25.891308 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:25.891499 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:25.891415 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:26.891552 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:26.891511 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:26.891992 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:26.891641 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:27.891635 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:27.891560 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:27.891635 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:27.891606 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:27.892081 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:27.891693 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:27.892081 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:27.891839 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:28.891845 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:28.891811 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:28.892242 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:28.891976 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:29.891169 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:29.891139 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:29.891355 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:29.891139 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:29.891355 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:29.891278 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:29.891355 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:29.891345 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:30.891239 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:30.891212 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:30.891657 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:30.891386 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:31.458063 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:31.458031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:31.458233 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.458120 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:31.458233 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.458163 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.458150872 +0000 UTC m=+33.142023587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:31.558823 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:31.558759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:31.558823 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:31.558804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:31.558967 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.558895 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:31.558967 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.558901 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:31.558967 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.558918 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:31.558967 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.558927 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:31.558967 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.558942 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.558929741 +0000 UTC m=+33.242802458 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:31.558967 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.558965 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.558953455 +0000 UTC m=+33.242826171 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:31.891089 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:31.891016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:31.891089 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:31.891016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:31.891290 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.891156 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:31.891290 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:31.891265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:32.891869 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:32.891833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:32.892285 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:32.891950 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:33.891781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:33.891748 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:33.891955 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:33.891882 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:33.891955 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:33.891944 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:33.892359 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:33.892086 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:34.893745 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:34.893720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:34.894118 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:34.893810 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:35.891263 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:35.891228 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:35.891451 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:35.891337 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:35.891451 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:35.891391 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:35.891564 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:35.891487 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:36.891713 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:36.891502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:36.892258 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:36.891805 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:37.098095 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.098066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qb2lp" event={"ID":"e9af9005-775c-4224-9567-909f4795d169","Type":"ContainerStarted","Data":"3a8fa211854bc90bdaf8a42136e25d42c9d0c754ef79c1c49dfeb2647944c116"} Jul 18 01:48:37.099911 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.099882 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lmppk" event={"ID":"58882a96-d369-4022-88bb-7c6222deacd6","Type":"ContainerStarted","Data":"ac32d2d68053938d1c242a851e497dc173406fc63de955b5bcc741e556c4d46d"} Jul 18 01:48:37.102933 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.102909 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" event={"ID":"14307c8ae8c30716921eb918c5798d1d","Type":"ContainerStarted","Data":"b270c45a0d07b52bcf41c1a6d4e328b73f16f0dd6a7757c63d166e46b6051596"} Jul 18 01:48:37.115377 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.115329 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qb2lp" podStartSLOduration=2.9392609419999998 podStartE2EDuration="23.115315839s" totalImagesPullingTime="20.176054897s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:37.114715256 +0000 UTC m=+22.798588020" watchObservedRunningTime="2026-07-18 01:48:37.115315839 +0000 UTC m=+22.799188577" Jul 18 01:48:37.131386 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.131336 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-56.ec2.internal" podStartSLOduration=22.131316647 podStartE2EDuration="22.131316647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:37.127756664 +0000 UTC m=+22.811629401" watchObservedRunningTime="2026-07-18 01:48:37.131316647 +0000 UTC m=+22.815189385" Jul 18 01:48:37.891530 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.891177 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:37.891708 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:37.891225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:37.891708 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:37.891550 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:37.891708 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:37.891644 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:38.106629 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.106573 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xsvd" event={"ID":"50f3cf71-8c8d-40af-8004-0b7886594448","Type":"ContainerStarted","Data":"59924aaf17a92bd6f896a29e6427285003f7673a58fa0969359bf2ef9a4f0dc6"} Jul 18 01:48:38.107978 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.107953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" event={"ID":"ca5dcb47-0098-492e-a219-40954bd79b4e","Type":"ContainerStarted","Data":"56639190cc1c2778c5db0fb4ef45c2ff001c69a26e065f854c2ecd08e2fa0d3e"} Jul 18 01:48:38.109337 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.109314 2580 generic.go:356] "Generic (PLEG): container finished" podID="2b6887dcbefc1c78a82854291c613ebc" containerID="945b14d0a10b3db3fb4f09d112f5c7d4f4f185e68e0f6ce5b49568009bde2a2c" exitCode=0 Jul 18 01:48:38.109421 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.109377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" event={"ID":"2b6887dcbefc1c78a82854291c613ebc","Type":"ContainerDied","Data":"945b14d0a10b3db3fb4f09d112f5c7d4f4f185e68e0f6ce5b49568009bde2a2c"} Jul 18 01:48:38.110884 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.110866 2580 generic.go:356] "Generic (PLEG): container finished" podID="a79aa2c5-6ecd-4434-a381-adc4351be14a" containerID="dfd38fd2afad6dfeac7430bbf2909a8bf4aa7eaa64d23098ad77d60027ef51bc" exitCode=0 Jul 18 01:48:38.110972 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.110930 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerDied","Data":"dfd38fd2afad6dfeac7430bbf2909a8bf4aa7eaa64d23098ad77d60027ef51bc"} Jul 18 01:48:38.112330 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.112310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87z6p" event={"ID":"891e9aab-d18d-4130-94b5-e1ec8af5717b","Type":"ContainerStarted","Data":"00edf56b43af25cb7f8fc634c7866a2f3a0f16b8804edfa1470610228a89480e"} Jul 18 01:48:38.113797 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.113776 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-966z9" event={"ID":"fb03f2c1-c046-416e-a6e6-85f56efe4964","Type":"ContainerStarted","Data":"f390b3f43a24277311e85223268fadebac03a2635029d17ecda7f4f6d54dacc1"} Jul 18 01:48:38.116349 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116333 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:48:38.116669 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116648 2580 generic.go:356] "Generic (PLEG): container finished" podID="880e0b4b-9383-434f-8338-9834f1240186" containerID="5527266df40ddd525d55774f30243f3f30395712eac287d6903ecc9ac64ae46e" exitCode=1 Jul 18 01:48:38.116744 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116715 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"fa2a0426cb4e1b8c0d798f0023b54076f569139c0d3fa3f0b645b00eb75090f6"} Jul 18 01:48:38.116792 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116765 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"35847877dea69f36f5b10620fcb042aa5c509400a1a92e6237d62e7662bc8874"} Jul 18 01:48:38.116840 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116828 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"15dc4f2405d34dcf51b242ebbf711339ac660ecf12be250420c43cc919c8a6c6"} Jul 18 01:48:38.116878 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"08073e0570305987876fa0d2c7110eeda5cf15b3a26017d6b497fb390daf30a5"} Jul 18 01:48:38.116927 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116896 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"fd0ff0b0482594af7f14e8cc2f809e65153d4e15c32391e6e6395604deeb72b3"} Jul 18 01:48:38.116973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.116929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerDied","Data":"5527266df40ddd525d55774f30243f3f30395712eac287d6903ecc9ac64ae46e"} Jul 18 01:48:38.117951 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.117928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nt7fw" event={"ID":"236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe","Type":"ContainerStarted","Data":"3cc67600be79e77aa2e912d8a7a51c4a9c4f5cb09c2c262c80299f9da075fc05"} Jul 18 01:48:38.120450 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.120415 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8xsvd" podStartSLOduration=4.265455998 podStartE2EDuration="24.120403266s" totalImagesPullingTime="19.854947268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.120344663 +0000 UTC m=+23.804217401" watchObservedRunningTime="2026-07-18 01:48:38.120403266 +0000 UTC m=+23.804276003" Jul 18 01:48:38.120741 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.120721 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lmppk" podStartSLOduration=3.104446448 podStartE2EDuration="23.12071549s" totalImagesPullingTime="20.016269042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:37.144352274 +0000 UTC m=+22.828225015" watchObservedRunningTime="2026-07-18 01:48:38.12071549 +0000 UTC m=+23.804588227" Jul 18 01:48:38.134245 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.134209 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-966z9" podStartSLOduration=4.105693746 podStartE2EDuration="24.134196703s" totalImagesPullingTime="20.028502957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.133938772 +0000 UTC m=+23.817811511" watchObservedRunningTime="2026-07-18 01:48:38.134196703 +0000 UTC m=+23.818069484" Jul 18 01:48:38.147240 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.147196 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-87z6p" podStartSLOduration=3.164434294 podStartE2EDuration="23.147182685s" totalImagesPullingTime="19.982748391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.147010255 +0000 UTC m=+23.830882984" watchObservedRunningTime="2026-07-18 01:48:38.147182685 +0000 UTC m=+23.831055428" Jul 18 01:48:38.197275 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.197078 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nt7fw" podStartSLOduration=4.671142931 podStartE2EDuration="24.197062452s" totalImagesPullingTime="19.525919521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.196624483 +0000 UTC m=+23.880497223" watchObservedRunningTime="2026-07-18 01:48:38.197062452 +0000 UTC m=+23.880935189" Jul 18 01:48:38.526721 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.526633 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 01:48:38.604045 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.604016 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:38.604694 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.604677 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:38.817708 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.817529 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:48:38.526655426Z","UUID":"86458925-cc8f-4346-b64d-54ee42dc0cfd","Handler":null,"Name":"","Endpoint":""} Jul 18 01:48:38.819077 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.819057 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 01:48:38.819206 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.819096 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 01:48:38.891737 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:38.891702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:38.892235 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:38.891833 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:39.123951 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.123865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" event={"ID":"ca5dcb47-0098-492e-a219-40954bd79b4e","Type":"ContainerStarted","Data":"ade212dd4373e07daf3bc8fb46aa0d5b304ff506543e877c0669ca14f80430df"} Jul 18 01:48:39.126088 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.126050 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" event={"ID":"2b6887dcbefc1c78a82854291c613ebc","Type":"ContainerStarted","Data":"0f99bfc0a0b874df77b4f8ba6c4ea79bed7bc4f2c20611bc44abed83866c19ff"} Jul 18 01:48:39.126549 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.126520 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:39.126847 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.126829 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nt7fw" Jul 18 01:48:39.152152 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.152081 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-56.ec2.internal" podStartSLOduration=24.152060654 podStartE2EDuration="24.152060654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:39.139751914 +0000 UTC m=+24.823624652" watchObservedRunningTime="2026-07-18 01:48:39.152060654 +0000 UTC m=+24.835933452" Jul 18 01:48:39.891539 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.891512 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:39.891704 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:39.891510 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:39.891704 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:39.891665 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:39.892092 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:39.891738 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:40.130738 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:40.130477 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" event={"ID":"ca5dcb47-0098-492e-a219-40954bd79b4e","Type":"ContainerStarted","Data":"cac0e057cc6199acc90d6c5bd824373938e7c441a169eb0ac7a0a71c8dbb0580"} Jul 18 01:48:40.134081 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:40.134056 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:48:40.134574 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:40.134548 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"1be37fafd3cbb814feef9e8da2ad1bff8ee822a6fed2741f16885ecd9ea170af"} Jul 18 01:48:40.147666 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:40.147606 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bjt4" podStartSLOduration=4.035946229 podStartE2EDuration="26.14757087s" totalImagesPullingTime="22.111624641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:40.146421849 +0000 UTC m=+25.830294601" watchObservedRunningTime="2026-07-18 01:48:40.14757087 +0000 UTC m=+25.831443610" Jul 18 01:48:40.891736 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:40.891690 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:40.891927 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:40.891838 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:41.891572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:41.891379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:41.891757 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:41.891421 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:41.891757 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:41.891683 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:41.891839 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:41.891746 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:42.894069 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:42.893989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:42.894745 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:42.894090 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:43.143363 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.143329 2580 generic.go:356] "Generic (PLEG): container finished" podID="a79aa2c5-6ecd-4434-a381-adc4351be14a" containerID="5a7061349a0fb3fd98f2d243ae95215989843983288a6ae2e64ecc335f28d703" exitCode=0 Jul 18 01:48:43.143642 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.143399 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerDied","Data":"5a7061349a0fb3fd98f2d243ae95215989843983288a6ae2e64ecc335f28d703"} Jul 18 01:48:43.147059 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.147038 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:48:43.147395 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.147366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"b0931eb75d4078e7efc47fb1abc53546295b62592571bbac57aba79d31ea497e"} Jul 18 01:48:43.147689 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.147665 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:43.147876 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.147859 2580 scope.go:117] "RemoveContainer" containerID="5527266df40ddd525d55774f30243f3f30395712eac287d6903ecc9ac64ae46e" Jul 18 01:48:43.164082 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.164058 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:43.892258 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.892066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:43.892562 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:43.892126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:43.892562 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:43.892341 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:43.892562 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:43.892393 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:44.153722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.153694 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:48:44.154122 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.154001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" event={"ID":"880e0b4b-9383-434f-8338-9834f1240186","Type":"ContainerStarted","Data":"0242dac094dfde234cd795dfbbb90262e33753f9da3b5e65f001782fd49d2be2"} Jul 18 01:48:44.154272 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.154253 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:44.154335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.154316 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:44.170807 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.170781 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:48:44.188246 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.188190 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" podStartSLOduration=9.212933713 podStartE2EDuration="30.188173987s" totalImagesPullingTime="20.975240274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:44.187421088 +0000 UTC m=+29.871293828" watchObservedRunningTime="2026-07-18 01:48:44.188173987 +0000 UTC m=+29.872046725" Jul 18 01:48:44.433504 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.433426 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4xj87"] Jul 18 01:48:44.433677 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.433540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:44.433677 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:44.433654 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:44.434001 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.433967 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rf5pp"] Jul 18 01:48:44.434117 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.434053 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:44.434178 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:44.434149 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:44.446424 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.446393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frd7c"] Jul 18 01:48:44.446601 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:44.446519 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:44.446679 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:44.446640 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:45.158706 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:45.158668 2580 generic.go:356] "Generic (PLEG): container finished" podID="a79aa2c5-6ecd-4434-a381-adc4351be14a" containerID="6500ef069ddd5a1e008afdef2efe9bba92e5db45afae38af4966f6d1ae297af4" exitCode=0 Jul 18 01:48:45.159227 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:45.158759 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerDied","Data":"6500ef069ddd5a1e008afdef2efe9bba92e5db45afae38af4966f6d1ae297af4"} Jul 18 01:48:45.892059 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:45.892028 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:45.892059 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:45.892047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:45.892310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:45.892132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:45.892310 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:45.892147 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:45.892310 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:45.892209 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:45.892310 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:45.892265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:47.165093 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.164907 2580 generic.go:356] "Generic (PLEG): container finished" podID="a79aa2c5-6ecd-4434-a381-adc4351be14a" containerID="fd7b5b54668e5f7d0655ab9cfd8a469d2aef8fe76328e959fa0ea91aa24fed09" exitCode=0 Jul 18 01:48:47.165093 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.164984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerDied","Data":"fd7b5b54668e5f7d0655ab9cfd8a469d2aef8fe76328e959fa0ea91aa24fed09"} Jul 18 01:48:47.467713 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.467623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:47.467847 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.467735 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:47.467847 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.467791 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs podName:19f50b49-b7d3-4df2-9619-4464c6bf21ba nodeName:}" failed. No retries permitted until 2026-07-18 01:49:19.467776218 +0000 UTC m=+65.151648933 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs") pod "network-metrics-daemon-4xj87" (UID: "19f50b49-b7d3-4df2-9619-4464c6bf21ba") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:47.568292 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.568254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:47.568457 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.568304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:47.568457 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.568400 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:47.568457 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.568407 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:47.568457 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.568429 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:47.568457 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.568439 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5m7t7 for pod openshift-network-diagnostics/network-check-target-frd7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:47.568457 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.568453 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret podName:bc49c9e2-a220-4c35-979c-0fd30a51cc5e nodeName:}" failed. No retries permitted until 2026-07-18 01:49:19.568439304 +0000 UTC m=+65.252312020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret") pod "global-pull-secret-syncer-rf5pp" (UID: "bc49c9e2-a220-4c35-979c-0fd30a51cc5e") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:47.568693 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.568489 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7 podName:6916c7b2-ecc9-4234-abbc-bd032374f743 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:19.568476945 +0000 UTC m=+65.252349662 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5m7t7" (UniqueName: "kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7") pod "network-check-target-frd7c" (UID: "6916c7b2-ecc9-4234-abbc-bd032374f743") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:47.891742 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.891656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:47.891742 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.891715 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:47.891962 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:47.891656 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:47.891962 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.891807 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:47.891962 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.891901 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:47.892135 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:47.891987 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:49.891729 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:49.891691 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:49.892209 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:49.891734 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:49.892209 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:49.891738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:49.892209 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:49.891820 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4xj87" podUID="19f50b49-b7d3-4df2-9619-4464c6bf21ba" Jul 18 01:48:49.892209 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:49.891915 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frd7c" podUID="6916c7b2-ecc9-4234-abbc-bd032374f743" Jul 18 01:48:49.892209 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:49.891985 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rf5pp" podUID="bc49c9e2-a220-4c35-979c-0fd30a51cc5e" Jul 18 01:48:50.143976 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.143904 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-56.ec2.internal" event="NodeReady" Jul 18 01:48:50.144124 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.144036 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 01:48:50.192443 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.192312 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b6h62"] Jul 18 01:48:50.194966 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.194944 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7db787ff66-nc6qk"] Jul 18 01:48:50.195105 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.195069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.197004 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.196983 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l8w46"] Jul 18 01:48:50.197128 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.197109 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.197636 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.197614 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 01:48:50.198118 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.197917 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zdfxb\"" Jul 18 01:48:50.198118 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.198057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 01:48:50.199133 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.199117 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9244r\"" Jul 18 01:48:50.199278 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.199171 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.199278 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.199265 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 01:48:50.199714 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.199696 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 01:48:50.199815 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.199806 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 01:48:50.201511 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.201485 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 01:48:50.201672 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.201641 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 01:48:50.203783 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.201904 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 01:48:50.204260 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.204236 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s99dg\"" Jul 18 01:48:50.206571 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.206551 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 01:48:50.208973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.208955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b6h62"] Jul 18 01:48:50.211234 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.211209 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7db787ff66-nc6qk"] Jul 18 01:48:50.212162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.212142 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l8w46"] Jul 18 01:48:50.290179 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t24nj\" (UniqueName: \"kubernetes.io/projected/7d8e622b-fcde-4197-b1da-670a26b2f3e4-kube-api-access-t24nj\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.290358 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-installation-pull-secrets\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290358 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-image-registry-private-configuration\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290358 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8e622b-fcde-4197-b1da-670a26b2f3e4-config-volume\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.290508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d8e622b-fcde-4197-b1da-670a26b2f3e4-tmp-dir\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.290508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-registry-certificates\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa8c1918-c53a-46bb-a01d-d508b6e8f7c2-cert\") pod \"ingress-canary-l8w46\" (UID: \"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2\") " pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.290508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2aff066-f708-41d8-89be-069131d7bc4f-ca-trust-extracted\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-bound-sa-token\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-trusted-ca\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.290725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvgdf\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-kube-api-access-cvgdf\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.290725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.290700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w25s\" (UniqueName: \"kubernetes.io/projected/aa8c1918-c53a-46bb-a01d-d508b6e8f7c2-kube-api-access-8w25s\") pod \"ingress-canary-l8w46\" (UID: \"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2\") " pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.391526 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t24nj\" (UniqueName: \"kubernetes.io/projected/7d8e622b-fcde-4197-b1da-670a26b2f3e4-kube-api-access-t24nj\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.391722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-installation-pull-secrets\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.391722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-image-registry-private-configuration\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.391722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8e622b-fcde-4197-b1da-670a26b2f3e4-config-volume\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.391722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d8e622b-fcde-4197-b1da-670a26b2f3e4-tmp-dir\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.391722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-registry-certificates\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.391722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa8c1918-c53a-46bb-a01d-d508b6e8f7c2-cert\") pod \"ingress-canary-l8w46\" (UID: \"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2\") " pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.392041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2aff066-f708-41d8-89be-069131d7bc4f-ca-trust-extracted\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.392041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391764 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-bound-sa-token\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.392041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.391803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.392041 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.391883 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:50.392041 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.391896 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7db787ff66-nc6qk: secret "image-registry-tls" not found Jul 18 01:48:50.392041 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.391954 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls podName:d2aff066-f708-41d8-89be-069131d7bc4f nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.891937839 +0000 UTC m=+36.575810564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls") pod "image-registry-7db787ff66-nc6qk" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f") : secret "image-registry-tls" not found Jul 18 01:48:50.392341 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7d8e622b-fcde-4197-b1da-670a26b2f3e4-tmp-dir\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.392341 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392266 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2aff066-f708-41d8-89be-069131d7bc4f-ca-trust-extracted\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.392442 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-trusted-ca\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.392442 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.392442 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvgdf\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-kube-api-access-cvgdf\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.392612 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392450 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7d8e622b-fcde-4197-b1da-670a26b2f3e4-config-volume\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.392612 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w25s\" (UniqueName: \"kubernetes.io/projected/aa8c1918-c53a-46bb-a01d-d508b6e8f7c2-kube-api-access-8w25s\") pod \"ingress-canary-l8w46\" (UID: \"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2\") " pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.392612 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.392555 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:50.392764 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.392654 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls podName:7d8e622b-fcde-4197-b1da-670a26b2f3e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.892638974 +0000 UTC m=+36.576511689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls") pod "dns-default-b6h62" (UID: "7d8e622b-fcde-4197-b1da-670a26b2f3e4") : secret "dns-default-metrics-tls" not found Jul 18 01:48:50.392764 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.392666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-registry-certificates\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.393290 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.393268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-trusted-ca\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.396411 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.396361 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-installation-pull-secrets\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.396411 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.396362 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-image-registry-private-configuration\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.396565 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.396538 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa8c1918-c53a-46bb-a01d-d508b6e8f7c2-cert\") pod \"ingress-canary-l8w46\" (UID: \"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2\") " pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.400132 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.400099 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t24nj\" (UniqueName: \"kubernetes.io/projected/7d8e622b-fcde-4197-b1da-670a26b2f3e4-kube-api-access-t24nj\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.401127 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.401103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-bound-sa-token\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.402227 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.402183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w25s\" (UniqueName: \"kubernetes.io/projected/aa8c1918-c53a-46bb-a01d-d508b6e8f7c2-kube-api-access-8w25s\") pod \"ingress-canary-l8w46\" (UID: \"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2\") " pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.414644 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.414571 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvgdf\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-kube-api-access-cvgdf\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.525966 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.525928 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l8w46" Jul 18 01:48:50.896366 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.896282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:50.896366 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:50.896330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:50.896904 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.896435 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:50.896904 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.896437 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:50.896904 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.896456 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7db787ff66-nc6qk: secret "image-registry-tls" not found Jul 18 01:48:50.896904 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.896492 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls podName:7d8e622b-fcde-4197-b1da-670a26b2f3e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.896477259 +0000 UTC m=+37.580349975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls") pod "dns-default-b6h62" (UID: "7d8e622b-fcde-4197-b1da-670a26b2f3e4") : secret "dns-default-metrics-tls" not found Jul 18 01:48:50.896904 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:50.896505 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls podName:d2aff066-f708-41d8-89be-069131d7bc4f nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.896499255 +0000 UTC m=+37.580371970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls") pod "image-registry-7db787ff66-nc6qk" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f") : secret "image-registry-tls" not found Jul 18 01:48:51.215295 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.215065 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l8w46"] Jul 18 01:48:51.219691 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:51.219645 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa8c1918_c53a_46bb_a01d_d508b6e8f7c2.slice/crio-f0d7063f52f02f6c77be513d37768f239c52d1353725e85e9bdbd16bd7a82b63 WatchSource:0}: Error finding container f0d7063f52f02f6c77be513d37768f239c52d1353725e85e9bdbd16bd7a82b63: Status 404 returned error can't find the container with id f0d7063f52f02f6c77be513d37768f239c52d1353725e85e9bdbd16bd7a82b63 Jul 18 01:48:51.891574 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.891541 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:48:51.891574 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.891542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:48:51.891832 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.891542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:48:51.894766 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.894738 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:48:51.894766 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.894755 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:48:51.894766 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.894738 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4m6k\"" Jul 18 01:48:51.894961 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.894800 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzg6q\"" Jul 18 01:48:51.894961 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.894762 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:48:51.894961 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.894745 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:48:51.906862 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.906829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:51.907480 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:51.906876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:51.907480 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:51.906984 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:51.907480 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:51.907041 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls podName:7d8e622b-fcde-4197-b1da-670a26b2f3e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:53.90702298 +0000 UTC m=+39.590895700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls") pod "dns-default-b6h62" (UID: "7d8e622b-fcde-4197-b1da-670a26b2f3e4") : secret "dns-default-metrics-tls" not found Jul 18 01:48:51.907480 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:51.906984 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:51.907480 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:51.907081 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7db787ff66-nc6qk: secret "image-registry-tls" not found Jul 18 01:48:51.907480 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:51.907136 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls podName:d2aff066-f708-41d8-89be-069131d7bc4f nodeName:}" failed. No retries permitted until 2026-07-18 01:48:53.907119863 +0000 UTC m=+39.590992590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls") pod "image-registry-7db787ff66-nc6qk" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f") : secret "image-registry-tls" not found Jul 18 01:48:52.179816 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:52.179731 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l8w46" event={"ID":"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2","Type":"ContainerStarted","Data":"f0d7063f52f02f6c77be513d37768f239c52d1353725e85e9bdbd16bd7a82b63"} Jul 18 01:48:52.182879 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:52.182844 2580 generic.go:356] "Generic (PLEG): container finished" podID="a79aa2c5-6ecd-4434-a381-adc4351be14a" containerID="9fe10960141a22e37e4ad822f8ea77d7f79a86ec518029c622687d275c7486fb" exitCode=0 Jul 18 01:48:52.182879 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:52.182878 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerDied","Data":"9fe10960141a22e37e4ad822f8ea77d7f79a86ec518029c622687d275c7486fb"} Jul 18 01:48:53.189718 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:53.189683 2580 generic.go:356] "Generic (PLEG): container finished" podID="a79aa2c5-6ecd-4434-a381-adc4351be14a" containerID="1a50f8ca766908227eb5d4b00eaec4e40d72d708cf1d6165ad9a2ae5ac6cf5b4" exitCode=0 Jul 18 01:48:53.190213 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:53.189750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerDied","Data":"1a50f8ca766908227eb5d4b00eaec4e40d72d708cf1d6165ad9a2ae5ac6cf5b4"} Jul 18 01:48:53.922872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:53.922615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:53.922872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:53.922846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:53.923087 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:53.922764 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:53.923087 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:53.922898 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7db787ff66-nc6qk: secret "image-registry-tls" not found Jul 18 01:48:53.923087 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:53.922946 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:53.923087 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:53.922962 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls podName:d2aff066-f708-41d8-89be-069131d7bc4f nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.922944089 +0000 UTC m=+43.606816807 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls") pod "image-registry-7db787ff66-nc6qk" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f") : secret "image-registry-tls" not found Jul 18 01:48:53.923087 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:53.922979 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls podName:7d8e622b-fcde-4197-b1da-670a26b2f3e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.922969284 +0000 UTC m=+43.606842014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls") pod "dns-default-b6h62" (UID: "7d8e622b-fcde-4197-b1da-670a26b2f3e4") : secret "dns-default-metrics-tls" not found Jul 18 01:48:54.193985 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:54.193890 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l8w46" event={"ID":"aa8c1918-c53a-46bb-a01d-d508b6e8f7c2","Type":"ContainerStarted","Data":"ec3ec70af93847634df8d1160a2d06fd490becb8fc1f58806dd62c187ac6b759"} Jul 18 01:48:54.197169 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:54.197140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" event={"ID":"a79aa2c5-6ecd-4434-a381-adc4351be14a","Type":"ContainerStarted","Data":"99e01e6a1821490f0aa195c3ad246295853351e2875f7ebf88249e41a9868650"} Jul 18 01:48:54.232759 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:54.232708 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dqb6d" podStartSLOduration=10.122768758 podStartE2EDuration="40.232693356s" totalImagesPullingTime="30.109924598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:54.232232324 +0000 UTC m=+39.916105062" watchObservedRunningTime="2026-07-18 01:48:54.232693356 +0000 UTC m=+39.916566094" Jul 18 01:48:54.232952 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:54.232931 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l8w46" podStartSLOduration=1.922448301 podStartE2EDuration="4.23292523s" totalImagesPullingTime="2.310476929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:54.211081948 +0000 UTC m=+39.894954689" watchObservedRunningTime="2026-07-18 01:48:54.23292523 +0000 UTC m=+39.916797969" Jul 18 01:48:56.659595 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.659548 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cwtbf"] Jul 18 01:48:56.689595 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.689546 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cwtbf"] Jul 18 01:48:56.689749 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.689709 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.691828 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.691803 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 01:48:56.691959 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.691910 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 01:48:56.692064 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.692046 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 01:48:56.692623 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.692598 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 01:48:56.692712 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.692627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6l5cz\"" Jul 18 01:48:56.845257 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.845222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afbf040c-2b74-48d7-a323-55739ec6a3f0-data-volume\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.845257 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.845258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfbdk\" (UniqueName: \"kubernetes.io/projected/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-api-access-tfbdk\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.845474 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.845308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afbf040c-2b74-48d7-a323-55739ec6a3f0-crio-socket\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.845474 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.845382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.845474 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.845410 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.945789 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.945701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afbf040c-2b74-48d7-a323-55739ec6a3f0-crio-socket\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.945789 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.945745 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.945789 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.945761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:56.945839 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:56.945864 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:56.945906 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.445893473 +0000 UTC m=+43.129766189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.945911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/afbf040c-2b74-48d7-a323-55739ec6a3f0-crio-socket\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:56.945919 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.445912066 +0000 UTC m=+43.129784782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : configmap "kube-rbac-proxy" not found Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.945949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afbf040c-2b74-48d7-a323-55739ec6a3f0-data-volume\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.946025 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.945969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfbdk\" (UniqueName: \"kubernetes.io/projected/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-api-access-tfbdk\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.946269 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.946221 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/afbf040c-2b74-48d7-a323-55739ec6a3f0-data-volume\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:56.956316 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:56.956287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfbdk\" (UniqueName: \"kubernetes.io/projected/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-api-access-tfbdk\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:57.450335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.450303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:57.450335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.450338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:57.450759 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.450445 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:57.450759 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.450466 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:57.450759 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.450516 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.450499016 +0000 UTC m=+44.134371732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:57.450759 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.450533 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.450524276 +0000 UTC m=+44.134396998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : configmap "kube-rbac-proxy" not found Jul 18 01:48:57.451152 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.451131 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb"] Jul 18 01:48:57.463998 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.463978 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb"] Jul 18 01:48:57.464084 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.464061 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" Jul 18 01:48:57.466251 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.466231 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 18 01:48:57.466973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.466948 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:57.467072 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.466976 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dbz2g\"" Jul 18 01:48:57.551472 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.551439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6z64\" (UniqueName: \"kubernetes.io/projected/2c18e87e-8502-4328-9d2b-8c9fa9feb8b6-kube-api-access-w6z64\") pod \"migrator-7cc9c7ffd7-zn2wb\" (UID: \"2c18e87e-8502-4328-9d2b-8c9fa9feb8b6\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" Jul 18 01:48:57.652437 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.652399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6z64\" (UniqueName: \"kubernetes.io/projected/2c18e87e-8502-4328-9d2b-8c9fa9feb8b6-kube-api-access-w6z64\") pod \"migrator-7cc9c7ffd7-zn2wb\" (UID: \"2c18e87e-8502-4328-9d2b-8c9fa9feb8b6\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" Jul 18 01:48:57.659953 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.659927 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6z64\" (UniqueName: \"kubernetes.io/projected/2c18e87e-8502-4328-9d2b-8c9fa9feb8b6-kube-api-access-w6z64\") pod \"migrator-7cc9c7ffd7-zn2wb\" (UID: \"2c18e87e-8502-4328-9d2b-8c9fa9feb8b6\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" Jul 18 01:48:57.773791 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.773699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" Jul 18 01:48:57.913691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.913657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb"] Jul 18 01:48:57.916695 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:57.916670 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c18e87e_8502_4328_9d2b_8c9fa9feb8b6.slice/crio-708b4eab702c7ab1e577468b76ebb0230a70b3974df46b15f5a7299fe5feae67 WatchSource:0}: Error finding container 708b4eab702c7ab1e577468b76ebb0230a70b3974df46b15f5a7299fe5feae67: Status 404 returned error can't find the container with id 708b4eab702c7ab1e577468b76ebb0230a70b3974df46b15f5a7299fe5feae67 Jul 18 01:48:57.955156 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.955127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:48:57.955291 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:57.955165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:48:57.955347 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.955297 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:57.955347 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.955297 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:57.955347 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.955321 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7db787ff66-nc6qk: secret "image-registry-tls" not found Jul 18 01:48:57.955483 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.955351 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls podName:7d8e622b-fcde-4197-b1da-670a26b2f3e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:05.95533376 +0000 UTC m=+51.639206491 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls") pod "dns-default-b6h62" (UID: "7d8e622b-fcde-4197-b1da-670a26b2f3e4") : secret "dns-default-metrics-tls" not found Jul 18 01:48:57.955483 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:57.955365 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls podName:d2aff066-f708-41d8-89be-069131d7bc4f nodeName:}" failed. No retries permitted until 2026-07-18 01:49:05.955359452 +0000 UTC m=+51.639232168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls") pod "image-registry-7db787ff66-nc6qk" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f") : secret "image-registry-tls" not found Jul 18 01:48:58.042409 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:58.042329 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xsvd_50f3cf71-8c8d-40af-8004-0b7886594448/dns-node-resolver/0.log" Jul 18 01:48:58.208392 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:58.208359 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" event={"ID":"2c18e87e-8502-4328-9d2b-8c9fa9feb8b6","Type":"ContainerStarted","Data":"708b4eab702c7ab1e577468b76ebb0230a70b3974df46b15f5a7299fe5feae67"} Jul 18 01:48:58.459150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:58.459064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:58.459150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:58.459111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:48:58.459365 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:58.459202 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:58.459365 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:58.459240 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:58.459365 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:58.459272 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:00.459255219 +0000 UTC m=+46.143127935 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : configmap "kube-rbac-proxy" not found Jul 18 01:48:58.459365 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:48:58.459292 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:00.459282693 +0000 UTC m=+46.143155412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:59.031023 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.030987 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-mfvpb"] Jul 18 01:48:59.050602 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.050553 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-mfvpb"] Jul 18 01:48:59.050745 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.050683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.052899 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.052862 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 18 01:48:59.053040 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.053006 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 18 01:48:59.053572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.053558 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 18 01:48:59.053695 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.053557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 18 01:48:59.053695 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.053630 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8lxz2\"" Jul 18 01:48:59.166113 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.166071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-signing-cabundle\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.166284 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.166219 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-signing-key\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.166284 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.166264 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xfmq\" (UniqueName: \"kubernetes.io/projected/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-kube-api-access-4xfmq\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.242268 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.242240 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87z6p_891e9aab-d18d-4130-94b5-e1ec8af5717b/node-ca/0.log" Jul 18 01:48:59.267670 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.267629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-signing-key\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.267670 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.267672 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xfmq\" (UniqueName: \"kubernetes.io/projected/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-kube-api-access-4xfmq\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.267879 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.267725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-signing-cabundle\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.268453 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.268427 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-signing-cabundle\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.270014 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.269995 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-signing-key\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.275796 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.275770 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xfmq\" (UniqueName: \"kubernetes.io/projected/e6b7cf4f-3a0f-4f06-9249-5cdecacdae12-kube-api-access-4xfmq\") pod \"service-ca-58598695f7-mfvpb\" (UID: \"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12\") " pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.363222 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.363136 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-mfvpb" Jul 18 01:48:59.545062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.545034 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-mfvpb"] Jul 18 01:48:59.555120 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:48:59.555088 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b7cf4f_3a0f_4f06_9249_5cdecacdae12.slice/crio-4671ba5db6f995fec4b1b5d7e780e602b4b05d108729ee06007f13c62afef05d WatchSource:0}: Error finding container 4671ba5db6f995fec4b1b5d7e780e602b4b05d108729ee06007f13c62afef05d: Status 404 returned error can't find the container with id 4671ba5db6f995fec4b1b5d7e780e602b4b05d108729ee06007f13c62afef05d Jul 18 01:48:59.843422 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:48:59.843387 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l8w46_aa8c1918-c53a-46bb-a01d-d508b6e8f7c2/serve-healthcheck-canary/0.log" Jul 18 01:49:00.214701 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:00.214613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-mfvpb" event={"ID":"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12","Type":"ContainerStarted","Data":"4671ba5db6f995fec4b1b5d7e780e602b4b05d108729ee06007f13c62afef05d"} Jul 18 01:49:00.216559 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:00.216527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" event={"ID":"2c18e87e-8502-4328-9d2b-8c9fa9feb8b6","Type":"ContainerStarted","Data":"6dff3404e2a8ab01a99da1826e8e660adff46864c0e3ceec3998d75fdfbe1014"} Jul 18 01:49:00.216713 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:00.216600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" event={"ID":"2c18e87e-8502-4328-9d2b-8c9fa9feb8b6","Type":"ContainerStarted","Data":"8ed9941d3f92443341de1723dad1277d35af528420f7713fb6c02cd9a0ca9014"} Jul 18 01:49:00.239409 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:00.239346 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-zn2wb" podStartSLOduration=1.756669727 podStartE2EDuration="3.23932409s" totalImagesPullingTime="1.482654363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:00.238173097 +0000 UTC m=+45.922045873" watchObservedRunningTime="2026-07-18 01:49:00.23932409 +0000 UTC m=+45.923196829" Jul 18 01:49:00.478885 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:00.478843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:00.479077 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:00.478893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:00.479077 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:00.478972 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:49:00.479077 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:00.479032 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:49:00.479077 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:00.479075 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:04.479054612 +0000 UTC m=+50.162927344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : configmap "kube-rbac-proxy" not found Jul 18 01:49:00.479308 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:00.479094 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:04.479085473 +0000 UTC m=+50.162958195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : secret "insights-runtime-extractor-tls" not found Jul 18 01:49:02.223104 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:02.223068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-mfvpb" event={"ID":"e6b7cf4f-3a0f-4f06-9249-5cdecacdae12","Type":"ContainerStarted","Data":"8b1453a18599e397b064dac07b5705eec82cb585aad15ba65230e19cb4c2d11c"} Jul 18 01:49:02.238648 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:02.238593 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-mfvpb" podStartSLOduration=1.182496211 podStartE2EDuration="3.238563013s" totalImagesPullingTime="2.056066802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:02.237728365 +0000 UTC m=+47.921601104" watchObservedRunningTime="2026-07-18 01:49:02.238563013 +0000 UTC m=+47.922435751" Jul 18 01:49:04.512401 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:04.512360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:04.512401 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:04.512404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:04.512855 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:04.512600 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:49:04.512855 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:04.512677 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls podName:afbf040c-2b74-48d7-a323-55739ec6a3f0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:12.512658955 +0000 UTC m=+58.196531671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cwtbf" (UID: "afbf040c-2b74-48d7-a323-55739ec6a3f0") : secret "insights-runtime-extractor-tls" not found Jul 18 01:49:04.512934 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:04.512898 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/afbf040c-2b74-48d7-a323-55739ec6a3f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:06.026434 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:06.026387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") pod \"image-registry-7db787ff66-nc6qk\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:49:06.026434 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:06.026439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:49:06.026908 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:06.026528 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:49:06.026908 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:06.026534 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:49:06.026908 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:06.026558 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7db787ff66-nc6qk: secret "image-registry-tls" not found Jul 18 01:49:06.026908 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:06.026599 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls podName:7d8e622b-fcde-4197-b1da-670a26b2f3e4 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.026567214 +0000 UTC m=+67.710439930 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls") pod "dns-default-b6h62" (UID: "7d8e622b-fcde-4197-b1da-670a26b2f3e4") : secret "dns-default-metrics-tls" not found Jul 18 01:49:06.026908 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:06.026626 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls podName:d2aff066-f708-41d8-89be-069131d7bc4f nodeName:}" failed. No retries permitted until 2026-07-18 01:49:22.026610163 +0000 UTC m=+67.710482889 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls") pod "image-registry-7db787ff66-nc6qk" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f") : secret "image-registry-tls" not found Jul 18 01:49:12.573703 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:12.573661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:12.575895 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:12.575873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/afbf040c-2b74-48d7-a323-55739ec6a3f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cwtbf\" (UID: \"afbf040c-2b74-48d7-a323-55739ec6a3f0\") " pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:12.598918 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:12.598868 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cwtbf" Jul 18 01:49:12.728481 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:12.728447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cwtbf"] Jul 18 01:49:12.731821 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:12.731793 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafbf040c_2b74_48d7_a323_55739ec6a3f0.slice/crio-1e0d73d447bf5a6e6b5b5498bed90ad9150bc8d0c5e4d73a2ee9f99fdbf61540 WatchSource:0}: Error finding container 1e0d73d447bf5a6e6b5b5498bed90ad9150bc8d0c5e4d73a2ee9f99fdbf61540: Status 404 returned error can't find the container with id 1e0d73d447bf5a6e6b5b5498bed90ad9150bc8d0c5e4d73a2ee9f99fdbf61540 Jul 18 01:49:13.254228 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:13.254119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cwtbf" event={"ID":"afbf040c-2b74-48d7-a323-55739ec6a3f0","Type":"ContainerStarted","Data":"1e0d73d447bf5a6e6b5b5498bed90ad9150bc8d0c5e4d73a2ee9f99fdbf61540"} Jul 18 01:49:16.178460 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:16.178421 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kcwzg" Jul 18 01:49:18.272055 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:18.272011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cwtbf" event={"ID":"afbf040c-2b74-48d7-a323-55739ec6a3f0","Type":"ContainerStarted","Data":"1f7a82c1d40fcfa6bbcdaedfb7162bad3cc14e8ab6ce07961fef40540e53cf0e"} Jul 18 01:49:19.147774 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.147738 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4"] Jul 18 01:49:19.150990 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.150750 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.153932 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.153912 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 18 01:49:19.154166 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.154148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 18 01:49:19.154973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.154952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-5kqs5\"" Jul 18 01:49:19.162410 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.162349 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4"] Jul 18 01:49:19.187160 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.187131 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7db787ff66-nc6qk"] Jul 18 01:49:19.187413 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:19.187384 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" podUID="d2aff066-f708-41d8-89be-069131d7bc4f" Jul 18 01:49:19.231883 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.231855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05432b85-ea07-4907-8d79-63c6d1393c1a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-gzhx4\" (UID: \"05432b85-ea07-4907-8d79-63c6d1393c1a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.231883 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.231888 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05432b85-ea07-4907-8d79-63c6d1393c1a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-gzhx4\" (UID: \"05432b85-ea07-4907-8d79-63c6d1393c1a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.254893 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.254858 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j"] Jul 18 01:49:19.258933 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.258900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.275863 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.275841 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:49:19.283096 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.283064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:49:19.284801 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.284740 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j"] Jul 18 01:49:19.332820 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.332785 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-installation-pull-secrets\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.332837 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-image-registry-private-configuration\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.332866 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-bound-sa-token\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.332897 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvgdf\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-kube-api-access-cvgdf\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.332924 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-trusted-ca\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.332960 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2aff066-f708-41d8-89be-069131d7bc4f-ca-trust-extracted\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333250 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333006 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-registry-certificates\") pod \"d2aff066-f708-41d8-89be-069131d7bc4f\" (UID: \"d2aff066-f708-41d8-89be-069131d7bc4f\") " Jul 18 01:49:19.333250 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-installation-pull-secrets\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333250 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-registry-tls\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333250 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-bound-sa-token\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333250 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333245 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-trusted-ca\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-image-registry-private-configuration\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-ca-trust-extracted\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-registry-certificates\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05432b85-ea07-4907-8d79-63c6d1393c1a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-gzhx4\" (UID: \"05432b85-ea07-4907-8d79-63c6d1393c1a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.333541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333487 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-247tf\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-kube-api-access-247tf\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.333541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333508 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:49:19.333868 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05432b85-ea07-4907-8d79-63c6d1393c1a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-gzhx4\" (UID: \"05432b85-ea07-4907-8d79-63c6d1393c1a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.333868 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333620 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-registry-certificates\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.333868 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333834 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2aff066-f708-41d8-89be-069131d7bc4f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:49:19.334026 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.333961 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:49:19.334617 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.334529 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/05432b85-ea07-4907-8d79-63c6d1393c1a-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-gzhx4\" (UID: \"05432b85-ea07-4907-8d79-63c6d1393c1a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.335515 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.335472 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:49:19.335762 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.335738 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:49:19.335911 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.335885 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-kube-api-access-cvgdf" (OuterVolumeSpecName: "kube-api-access-cvgdf") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "kube-api-access-cvgdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:49:19.336157 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.336121 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d2aff066-f708-41d8-89be-069131d7bc4f" (UID: "d2aff066-f708-41d8-89be-069131d7bc4f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:49:19.336295 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.336276 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05432b85-ea07-4907-8d79-63c6d1393c1a-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-gzhx4\" (UID: \"05432b85-ea07-4907-8d79-63c6d1393c1a\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.434809 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.434741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-trusted-ca\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.434952 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.434864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-image-registry-private-configuration\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.434952 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.434895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-ca-trust-extracted\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.434952 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.434924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-registry-certificates\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435111 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.434953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-247tf\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-kube-api-access-247tf\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435111 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.434999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-installation-pull-secrets\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435111 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435025 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-registry-tls\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435111 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-bound-sa-token\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435113 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-installation-pull-secrets\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.435300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435127 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-bound-sa-token\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.435300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435145 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d2aff066-f708-41d8-89be-069131d7bc4f-ca-trust-extracted\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.435300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435161 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d2aff066-f708-41d8-89be-069131d7bc4f-image-registry-private-configuration\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.435300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435176 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvgdf\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-kube-api-access-cvgdf\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.435300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435190 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d2aff066-f708-41d8-89be-069131d7bc4f-trusted-ca\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:19.435599 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-ca-trust-extracted\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435718 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435676 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-trusted-ca\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.435891 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.435793 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-registry-certificates\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.437598 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.437562 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-image-registry-private-configuration\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.437812 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.437790 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-registry-tls\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.437884 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.437831 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-installation-pull-secrets\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.447974 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.447956 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-bound-sa-token\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.448313 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.448288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-247tf\" (UniqueName: \"kubernetes.io/projected/5c2d9f85-0c6e-4a1a-8359-20464a09bddb-kube-api-access-247tf\") pod \"image-registry-7c5d7dd6cb-qtz9j\" (UID: \"5c2d9f85-0c6e-4a1a-8359-20464a09bddb\") " pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.460930 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.460907 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" Jul 18 01:49:19.536554 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.536519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:49:19.538942 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.538914 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:49:19.549808 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.549783 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19f50b49-b7d3-4df2-9619-4464c6bf21ba-metrics-certs\") pod \"network-metrics-daemon-4xj87\" (UID: \"19f50b49-b7d3-4df2-9619-4464c6bf21ba\") " pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:49:19.570031 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.570005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9244r\"" Jul 18 01:49:19.577862 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.577839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:19.637238 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.637204 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:49:19.637417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.637310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:49:19.639370 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.639338 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:49:19.639494 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.639478 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:49:19.650108 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.650089 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:49:19.650448 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.650428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc49c9e2-a220-4c35-979c-0fd30a51cc5e-original-pull-secret\") pod \"global-pull-secret-syncer-rf5pp\" (UID: \"bc49c9e2-a220-4c35-979c-0fd30a51cc5e\") " pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:49:19.661037 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.661017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m7t7\" (UniqueName: \"kubernetes.io/projected/6916c7b2-ecc9-4234-abbc-bd032374f743-kube-api-access-5m7t7\") pod \"network-check-target-frd7c\" (UID: \"6916c7b2-ecc9-4234-abbc-bd032374f743\") " pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:49:19.803704 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.803668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rf5pp" Jul 18 01:49:19.813646 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.813625 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4m6k\"" Jul 18 01:49:19.819684 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.819663 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzg6q\"" Jul 18 01:49:19.822216 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.822196 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4xj87" Jul 18 01:49:19.828650 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:19.828627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:49:20.279148 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:20.279119 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7db787ff66-nc6qk" Jul 18 01:49:20.329685 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:20.329653 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7db787ff66-nc6qk"] Jul 18 01:49:20.337646 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:20.337622 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7db787ff66-nc6qk"] Jul 18 01:49:20.443324 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:20.443291 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d2aff066-f708-41d8-89be-069131d7bc4f-registry-tls\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:49:20.895346 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:20.895311 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2aff066-f708-41d8-89be-069131d7bc4f" path="/var/lib/kubelet/pods/d2aff066-f708-41d8-89be-069131d7bc4f/volumes" Jul 18 01:49:21.219176 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.219146 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rf5pp"] Jul 18 01:49:21.222673 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:21.222643 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc49c9e2_a220_4c35_979c_0fd30a51cc5e.slice/crio-f5e139f43d5c37d411d8690a5f720cd044b9a9c9140df5b8b00ea147c932c78a WatchSource:0}: Error finding container f5e139f43d5c37d411d8690a5f720cd044b9a9c9140df5b8b00ea147c932c78a: Status 404 returned error can't find the container with id f5e139f43d5c37d411d8690a5f720cd044b9a9c9140df5b8b00ea147c932c78a Jul 18 01:49:21.285722 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.285690 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rf5pp" event={"ID":"bc49c9e2-a220-4c35-979c-0fd30a51cc5e","Type":"ContainerStarted","Data":"f5e139f43d5c37d411d8690a5f720cd044b9a9c9140df5b8b00ea147c932c78a"} Jul 18 01:49:21.287303 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.287280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cwtbf" event={"ID":"afbf040c-2b74-48d7-a323-55739ec6a3f0","Type":"ContainerStarted","Data":"c1d7a79671ea00bcbbbcbf0d85b6eb511b7d34e02cf6b943091204a48fa04514"} Jul 18 01:49:21.440184 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.440100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frd7c"] Jul 18 01:49:21.443417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.443142 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4"] Jul 18 01:49:21.443417 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:21.443270 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6916c7b2_ecc9_4234_abbc_bd032374f743.slice/crio-41531a41a179b721e719ee667b399da02404f4cc751124604a673577c71295e1 WatchSource:0}: Error finding container 41531a41a179b721e719ee667b399da02404f4cc751124604a673577c71295e1: Status 404 returned error can't find the container with id 41531a41a179b721e719ee667b399da02404f4cc751124604a673577c71295e1 Jul 18 01:49:21.444715 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.444311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4xj87"] Jul 18 01:49:21.446961 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:21.446815 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j"] Jul 18 01:49:21.448710 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:21.448446 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f50b49_b7d3_4df2_9619_4464c6bf21ba.slice/crio-2b37c57463b0286c02f1081ede37437b37d862a995b0781088b06ef01cd5b39c WatchSource:0}: Error finding container 2b37c57463b0286c02f1081ede37437b37d862a995b0781088b06ef01cd5b39c: Status 404 returned error can't find the container with id 2b37c57463b0286c02f1081ede37437b37d862a995b0781088b06ef01cd5b39c Jul 18 01:49:21.449304 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:21.449270 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05432b85_ea07_4907_8d79_63c6d1393c1a.slice/crio-7aeeebbbfe3f3140ed68b96630f5fda0da34ecc37de5ec2f483b1eb3d115a312 WatchSource:0}: Error finding container 7aeeebbbfe3f3140ed68b96630f5fda0da34ecc37de5ec2f483b1eb3d115a312: Status 404 returned error can't find the container with id 7aeeebbbfe3f3140ed68b96630f5fda0da34ecc37de5ec2f483b1eb3d115a312 Jul 18 01:49:21.450523 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:21.450500 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2d9f85_0c6e_4a1a_8359_20464a09bddb.slice/crio-3938be285e0bc9ce8e40ebca3ab03c94facc8adeafb2ce3738c9dd4ccb931681 WatchSource:0}: Error finding container 3938be285e0bc9ce8e40ebca3ab03c94facc8adeafb2ce3738c9dd4ccb931681: Status 404 returned error can't find the container with id 3938be285e0bc9ce8e40ebca3ab03c94facc8adeafb2ce3738c9dd4ccb931681 Jul 18 01:49:22.057300 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.057260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:49:22.070799 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.070711 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7d8e622b-fcde-4197-b1da-670a26b2f3e4-metrics-tls\") pod \"dns-default-b6h62\" (UID: \"7d8e622b-fcde-4197-b1da-670a26b2f3e4\") " pod="openshift-dns/dns-default-b6h62" Jul 18 01:49:22.294105 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.294067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xj87" event={"ID":"19f50b49-b7d3-4df2-9619-4464c6bf21ba","Type":"ContainerStarted","Data":"2b37c57463b0286c02f1081ede37437b37d862a995b0781088b06ef01cd5b39c"} Jul 18 01:49:22.296727 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.296697 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" event={"ID":"05432b85-ea07-4907-8d79-63c6d1393c1a","Type":"ContainerStarted","Data":"7aeeebbbfe3f3140ed68b96630f5fda0da34ecc37de5ec2f483b1eb3d115a312"} Jul 18 01:49:22.299042 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.299011 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" event={"ID":"5c2d9f85-0c6e-4a1a-8359-20464a09bddb","Type":"ContainerStarted","Data":"71d39f3850ba9498b8b60a45b08e24a1b2475f064926ac673e2e90e9388203c7"} Jul 18 01:49:22.299141 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.299067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" event={"ID":"5c2d9f85-0c6e-4a1a-8359-20464a09bddb","Type":"ContainerStarted","Data":"3938be285e0bc9ce8e40ebca3ab03c94facc8adeafb2ce3738c9dd4ccb931681"} Jul 18 01:49:22.299836 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.299816 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:22.309820 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.309669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frd7c" event={"ID":"6916c7b2-ecc9-4234-abbc-bd032374f743","Type":"ContainerStarted","Data":"41531a41a179b721e719ee667b399da02404f4cc751124604a673577c71295e1"} Jul 18 01:49:22.313523 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.313496 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zdfxb\"" Jul 18 01:49:22.321638 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.321612 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6h62" Jul 18 01:49:22.324005 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.323692 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" podStartSLOduration=3.323671857 podStartE2EDuration="3.323671857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:22.32223471 +0000 UTC m=+68.006107450" watchObservedRunningTime="2026-07-18 01:49:22.323671857 +0000 UTC m=+68.007544620" Jul 18 01:49:22.529503 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:22.529438 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b6h62"] Jul 18 01:49:23.421077 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:23.421033 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8e622b_fcde_4197_b1da_670a26b2f3e4.slice/crio-471db5fd8527e287b22150048a653ece416a5e0d2f4a166076ef7b7c47103a81 WatchSource:0}: Error finding container 471db5fd8527e287b22150048a653ece416a5e0d2f4a166076ef7b7c47103a81: Status 404 returned error can't find the container with id 471db5fd8527e287b22150048a653ece416a5e0d2f4a166076ef7b7c47103a81 Jul 18 01:49:24.320642 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:24.320604 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6h62" event={"ID":"7d8e622b-fcde-4197-b1da-670a26b2f3e4","Type":"ContainerStarted","Data":"471db5fd8527e287b22150048a653ece416a5e0d2f4a166076ef7b7c47103a81"} Jul 18 01:49:24.322268 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:24.322224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xj87" event={"ID":"19f50b49-b7d3-4df2-9619-4464c6bf21ba","Type":"ContainerStarted","Data":"db77c71f570677bfbbbef519e27cccf86f28d116c4775607164f5a077946e99d"} Jul 18 01:49:24.323863 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:24.323841 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" event={"ID":"05432b85-ea07-4907-8d79-63c6d1393c1a","Type":"ContainerStarted","Data":"3ce57396c6d5b2d9c20646f05a664e8dba34139f88ea5d6be9a63b24213f265a"} Jul 18 01:49:24.340417 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:24.340367 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-gzhx4" podStartSLOduration=3.020093221 podStartE2EDuration="5.34034868s" totalImagesPullingTime="2.320255459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:24.339338261 +0000 UTC m=+70.023211002" watchObservedRunningTime="2026-07-18 01:49:24.34034868 +0000 UTC m=+70.024221417" Jul 18 01:49:29.349764 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.349724 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cwtbf" event={"ID":"afbf040c-2b74-48d7-a323-55739ec6a3f0","Type":"ContainerStarted","Data":"f8f26ca612102bcb2c1b7b35ff2feb77e67b003f2cf71aec3246010b97668490"} Jul 18 01:49:29.351636 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.351601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4xj87" event={"ID":"19f50b49-b7d3-4df2-9619-4464c6bf21ba","Type":"ContainerStarted","Data":"2438ad48c266522cd3a6b8f1d16a8817d105f98607cb55f774022b95b36fecd9"} Jul 18 01:49:29.352840 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.352806 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rf5pp" event={"ID":"bc49c9e2-a220-4c35-979c-0fd30a51cc5e","Type":"ContainerStarted","Data":"ccb9df5b41ea0db892a636651b705148a5f6b31eba87ca721037c07426bcc26e"} Jul 18 01:49:29.354331 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.354307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6h62" event={"ID":"7d8e622b-fcde-4197-b1da-670a26b2f3e4","Type":"ContainerStarted","Data":"2bfdf7be6ab3e5a95eface0cf718a92f78e8b9cafa83b79fb2e1d2879c2ce116"} Jul 18 01:49:29.354452 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.354351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6h62" event={"ID":"7d8e622b-fcde-4197-b1da-670a26b2f3e4","Type":"ContainerStarted","Data":"8010af5912f4b69ad5af8a6b0a47339fa691ca8bb97497287fb747b57eff2fe4"} Jul 18 01:49:29.354452 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.354379 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b6h62" Jul 18 01:49:29.355572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.355551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frd7c" event={"ID":"6916c7b2-ecc9-4234-abbc-bd032374f743","Type":"ContainerStarted","Data":"c8be333065ec576fc5368de7c5b90c3f0f0806a5854eed47fa72e8d4c3e71ff3"} Jul 18 01:49:29.355691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.355677 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:49:29.369069 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.369031 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cwtbf" podStartSLOduration=17.706468893 podStartE2EDuration="33.369018709s" totalImagesPullingTime="15.662549816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:29.368257933 +0000 UTC m=+75.052130672" watchObservedRunningTime="2026-07-18 01:49:29.369018709 +0000 UTC m=+75.052891446" Jul 18 01:49:29.402825 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.402773 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4xj87" podStartSLOduration=73.083280773 podStartE2EDuration="1m15.402756301s" totalImagesPullingTime="2.319475528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:29.402070195 +0000 UTC m=+75.085942934" watchObservedRunningTime="2026-07-18 01:49:29.402756301 +0000 UTC m=+75.086629038" Jul 18 01:49:29.402965 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.402915 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rf5pp" podStartSLOduration=67.03357404 podStartE2EDuration="1m14.402906974s" totalImagesPullingTime="7.369332934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:29.383017982 +0000 UTC m=+75.066890720" watchObservedRunningTime="2026-07-18 01:49:29.402906974 +0000 UTC m=+75.086779715" Jul 18 01:49:29.417558 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.417507 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-frd7c" podStartSLOduration=67.270295439 podStartE2EDuration="1m14.417489703s" totalImagesPullingTime="7.147194264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:29.417418925 +0000 UTC m=+75.101291686" watchObservedRunningTime="2026-07-18 01:49:29.417489703 +0000 UTC m=+75.101362442" Jul 18 01:49:29.434683 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:29.434637 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b6h62" podStartSLOduration=34.547233873 podStartE2EDuration="39.434622557s" totalImagesPullingTime="4.887388684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:29.433891455 +0000 UTC m=+75.117764195" watchObservedRunningTime="2026-07-18 01:49:29.434622557 +0000 UTC m=+75.118495294" Jul 18 01:49:34.913637 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.913600 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng"] Jul 18 01:49:34.916540 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.916518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:34.918632 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.918607 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ml7g9\"" Jul 18 01:49:34.918845 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.918832 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 01:49:34.919150 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.919113 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 18 01:49:34.919444 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.919429 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 18 01:49:34.919563 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.919456 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 01:49:34.919743 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.919729 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 01:49:34.928367 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.928348 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng"] Jul 18 01:49:34.956991 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.956957 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rgqvw"] Jul 18 01:49:34.962391 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.962372 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:34.964231 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.964214 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 01:49:34.964326 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.964256 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6ncsh\"" Jul 18 01:49:34.964378 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.964334 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 01:49:34.964539 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:34.964526 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 01:49:35.056164 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2602a2ec-ebd8-4034-af72-d6a63cb985d9-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.056164 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-wtmp\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056222 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-tls\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-root\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056286 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602a2ec-ebd8-4034-af72-d6a63cb985d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnh5v\" (UniqueName: \"kubernetes.io/projected/339b185b-a593-46e2-9d17-3d8b01689e39-kube-api-access-tnh5v\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-accelerators-collector-config\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-textfile\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/339b185b-a593-46e2-9d17-3d8b01689e39-metrics-client-ca\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-sys\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.056419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2602a2ec-ebd8-4034-af72-d6a63cb985d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.056973 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.056431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4mv6\" (UniqueName: \"kubernetes.io/projected/2602a2ec-ebd8-4034-af72-d6a63cb985d9-kube-api-access-t4mv6\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.157731 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-tls\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-root\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602a2ec-ebd8-4034-af72-d6a63cb985d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnh5v\" (UniqueName: \"kubernetes.io/projected/339b185b-a593-46e2-9d17-3d8b01689e39-kube-api-access-tnh5v\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-accelerators-collector-config\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:35.157847 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-textfile\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157875 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-root\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.157925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.157885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/339b185b-a593-46e2-9d17-3d8b01689e39-metrics-client-ca\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:35.157931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-tls podName:339b185b-a593-46e2-9d17-3d8b01689e39 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:35.657910333 +0000 UTC m=+81.341783049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-tls") pod "node-exporter-rgqvw" (UID: "339b185b-a593-46e2-9d17-3d8b01689e39") : secret "node-exporter-tls" not found Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-sys\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2602a2ec-ebd8-4034-af72-d6a63cb985d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mv6\" (UniqueName: \"kubernetes.io/projected/2602a2ec-ebd8-4034-af72-d6a63cb985d9-kube-api-access-t4mv6\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2602a2ec-ebd8-4034-af72-d6a63cb985d9-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-wtmp\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158240 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-textfile\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158310 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-sys\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158784 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158405 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-wtmp\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.158784 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158512 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-accelerators-collector-config\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.159006 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158961 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2602a2ec-ebd8-4034-af72-d6a63cb985d9-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.159006 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.158977 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/339b185b-a593-46e2-9d17-3d8b01689e39-metrics-client-ca\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.160642 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.160624 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602a2ec-ebd8-4034-af72-d6a63cb985d9-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.160977 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.160957 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.161041 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.161014 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2602a2ec-ebd8-4034-af72-d6a63cb985d9-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.169099 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.169032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mv6\" (UniqueName: \"kubernetes.io/projected/2602a2ec-ebd8-4034-af72-d6a63cb985d9-kube-api-access-t4mv6\") pod \"openshift-state-metrics-79f6958cdf-lvwng\" (UID: \"2602a2ec-ebd8-4034-af72-d6a63cb985d9\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.169202 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.169143 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnh5v\" (UniqueName: \"kubernetes.io/projected/339b185b-a593-46e2-9d17-3d8b01689e39-kube-api-access-tnh5v\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.227273 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.227243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" Jul 18 01:49:35.373974 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.373891 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng"] Jul 18 01:49:35.378166 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:35.378139 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2602a2ec_ebd8_4034_af72_d6a63cb985d9.slice/crio-1c33abffb07cca7c30a168ba1ebc20b42041891a7118032b6031b7aa78521440 WatchSource:0}: Error finding container 1c33abffb07cca7c30a168ba1ebc20b42041891a7118032b6031b7aa78521440: Status 404 returned error can't find the container with id 1c33abffb07cca7c30a168ba1ebc20b42041891a7118032b6031b7aa78521440 Jul 18 01:49:35.662377 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.662347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-tls\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.664682 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.664661 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/339b185b-a593-46e2-9d17-3d8b01689e39-node-exporter-tls\") pod \"node-exporter-rgqvw\" (UID: \"339b185b-a593-46e2-9d17-3d8b01689e39\") " pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.873576 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:35.873542 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rgqvw" Jul 18 01:49:35.885520 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:35.885496 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339b185b_a593_46e2_9d17_3d8b01689e39.slice/crio-8bfc0f966ad5da01deff51044243f896cf5af0665563f223106cbddd7c20ec22 WatchSource:0}: Error finding container 8bfc0f966ad5da01deff51044243f896cf5af0665563f223106cbddd7c20ec22: Status 404 returned error can't find the container with id 8bfc0f966ad5da01deff51044243f896cf5af0665563f223106cbddd7c20ec22 Jul 18 01:49:36.380897 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:36.380865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rgqvw" event={"ID":"339b185b-a593-46e2-9d17-3d8b01689e39","Type":"ContainerStarted","Data":"8bfc0f966ad5da01deff51044243f896cf5af0665563f223106cbddd7c20ec22"} Jul 18 01:49:36.382592 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:36.382507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" event={"ID":"2602a2ec-ebd8-4034-af72-d6a63cb985d9","Type":"ContainerStarted","Data":"940976b58c679436bc2508c32ee9c946e2b8cf466fbb281e1dcdfa76b534622a"} Jul 18 01:49:36.382724 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:36.382629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" event={"ID":"2602a2ec-ebd8-4034-af72-d6a63cb985d9","Type":"ContainerStarted","Data":"4b0e4f1218ad738621e70f51b414cd12cdd06bf5af99c9fcdde74a892029d672"} Jul 18 01:49:36.382724 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:36.382661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" event={"ID":"2602a2ec-ebd8-4034-af72-d6a63cb985d9","Type":"ContainerStarted","Data":"1c33abffb07cca7c30a168ba1ebc20b42041891a7118032b6031b7aa78521440"} Jul 18 01:49:37.386842 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:37.386802 2580 generic.go:356] "Generic (PLEG): container finished" podID="339b185b-a593-46e2-9d17-3d8b01689e39" containerID="2aaa6e7defd3fdf21bcb6fb0a704e5768418be96d0755ae0697411143dd1a477" exitCode=0 Jul 18 01:49:37.387242 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:37.386887 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rgqvw" event={"ID":"339b185b-a593-46e2-9d17-3d8b01689e39","Type":"ContainerDied","Data":"2aaa6e7defd3fdf21bcb6fb0a704e5768418be96d0755ae0697411143dd1a477"} Jul 18 01:49:37.391137 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:37.391102 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" event={"ID":"2602a2ec-ebd8-4034-af72-d6a63cb985d9","Type":"ContainerStarted","Data":"e70e369b53a2c89b18a76ac859a29cb4e6d57a39a4443d205ca8552a571b340f"} Jul 18 01:49:37.425492 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:37.424722 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lvwng" podStartSLOduration=2.554550987 podStartE2EDuration="3.424697388s" totalImagesPullingTime="870.146401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:37.423013 +0000 UTC m=+83.106885739" watchObservedRunningTime="2026-07-18 01:49:37.424697388 +0000 UTC m=+83.108570127" Jul 18 01:49:38.397354 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:38.397315 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rgqvw" event={"ID":"339b185b-a593-46e2-9d17-3d8b01689e39","Type":"ContainerStarted","Data":"37b66668dc74c490918f5101c5b4e1e2219417f033f57acd0bc0c5190a31d041"} Jul 18 01:49:38.397776 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:38.397405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rgqvw" event={"ID":"339b185b-a593-46e2-9d17-3d8b01689e39","Type":"ContainerStarted","Data":"cbc4ccbb35d3ba28e57cf8252b616b11f10c63c5a34e1166682e5457cab157c9"} Jul 18 01:49:38.415118 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:38.415058 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rgqvw" podStartSLOduration=3.345813043 podStartE2EDuration="4.415035974s" totalImagesPullingTime="1.003937758s" totalInitContainerRuntime="65.285173ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:38.414726775 +0000 UTC m=+84.098599526" watchObservedRunningTime="2026-07-18 01:49:38.415035974 +0000 UTC m=+84.098908713" Jul 18 01:49:39.361335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.361306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b6h62" Jul 18 01:49:39.582419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.582383 2580 patch_prober.go:28] interesting pod/image-registry-7c5d7dd6cb-qtz9j container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 18 01:49:39.582788 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.582436 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" podUID="5c2d9f85-0c6e-4a1a-8359-20464a09bddb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 01:49:39.691087 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.691006 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk"] Jul 18 01:49:39.694598 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.694565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:39.696823 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.696803 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 18 01:49:39.696939 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.696913 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wrk5g\"" Jul 18 01:49:39.707545 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.707522 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk"] Jul 18 01:49:39.794010 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.793981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rfj2\" (UniqueName: \"kubernetes.io/projected/baaefe17-d29b-4b4c-86e0-ca6692d057b9-kube-api-access-7rfj2\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:39.794178 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.794027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaefe17-d29b-4b4c-86e0-ca6692d057b9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:39.894955 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.894928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rfj2\" (UniqueName: \"kubernetes.io/projected/baaefe17-d29b-4b4c-86e0-ca6692d057b9-kube-api-access-7rfj2\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:39.895114 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.894971 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaefe17-d29b-4b4c-86e0-ca6692d057b9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:39.895114 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:39.895066 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 18 01:49:39.895228 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:49:39.895124 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaefe17-d29b-4b4c-86e0-ca6692d057b9-monitoring-plugin-cert podName:baaefe17-d29b-4b4c-86e0-ca6692d057b9 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:40.395105026 +0000 UTC m=+86.078977747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/baaefe17-d29b-4b4c-86e0-ca6692d057b9-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-4qrxk" (UID: "baaefe17-d29b-4b4c-86e0-ca6692d057b9") : secret "monitoring-plugin-cert" not found Jul 18 01:49:39.903332 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:39.903308 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rfj2\" (UniqueName: \"kubernetes.io/projected/baaefe17-d29b-4b4c-86e0-ca6692d057b9-kube-api-access-7rfj2\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:40.397928 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:40.397890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaefe17-d29b-4b4c-86e0-ca6692d057b9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:40.400636 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:40.400616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/baaefe17-d29b-4b4c-86e0-ca6692d057b9-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-4qrxk\" (UID: \"baaefe17-d29b-4b4c-86e0-ca6692d057b9\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:40.604155 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:40.604102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:40.730765 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:40.727504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk"] Jul 18 01:49:41.410449 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:41.410407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" event={"ID":"baaefe17-d29b-4b4c-86e0-ca6692d057b9","Type":"ContainerStarted","Data":"a3a15dd48e5436ae8a02d87724d11cdba052ea79357f651e01042db6d2dbdc68"} Jul 18 01:49:42.416100 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:42.416064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" event={"ID":"baaefe17-d29b-4b4c-86e0-ca6692d057b9","Type":"ContainerStarted","Data":"bb5f08eab8b8ffc770129b046addb2dd023d203ce6c3c0ffdff39a89744c5d4e"} Jul 18 01:49:42.416467 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:42.416265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:42.420963 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:42.420939 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" Jul 18 01:49:42.433129 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:42.433089 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-4qrxk" podStartSLOduration=2.340236447 podStartE2EDuration="3.433077006s" totalImagesPullingTime="1.092840559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:42.431870511 +0000 UTC m=+88.115743249" watchObservedRunningTime="2026-07-18 01:49:42.433077006 +0000 UTC m=+88.116949804" Jul 18 01:49:44.328616 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.328570 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c5d7dd6cb-qtz9j" Jul 18 01:49:44.693572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.693492 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-7crkn"] Jul 18 01:49:44.696546 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.696530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:49:44.698807 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.698785 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 18 01:49:44.698912 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.698876 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 18 01:49:44.698912 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.698876 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-27fch\"" Jul 18 01:49:44.710697 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.710666 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-7crkn"] Jul 18 01:49:44.832755 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.832725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tshw\" (UniqueName: \"kubernetes.io/projected/725cba84-708c-47ff-ab2b-8d54c646ec0e-kube-api-access-9tshw\") pod \"downloads-5bc67dcd46-7crkn\" (UID: \"725cba84-708c-47ff-ab2b-8d54c646ec0e\") " pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:49:44.933827 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.933795 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tshw\" (UniqueName: \"kubernetes.io/projected/725cba84-708c-47ff-ab2b-8d54c646ec0e-kube-api-access-9tshw\") pod \"downloads-5bc67dcd46-7crkn\" (UID: \"725cba84-708c-47ff-ab2b-8d54c646ec0e\") " pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:49:44.941402 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:44.941374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tshw\" (UniqueName: \"kubernetes.io/projected/725cba84-708c-47ff-ab2b-8d54c646ec0e-kube-api-access-9tshw\") pod \"downloads-5bc67dcd46-7crkn\" (UID: \"725cba84-708c-47ff-ab2b-8d54c646ec0e\") " pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:49:45.005792 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:45.005761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:49:45.133042 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:45.133016 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-7crkn"] Jul 18 01:49:45.135597 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:45.135560 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725cba84_708c_47ff_ab2b_8d54c646ec0e.slice/crio-06a10996efb0b022fac0edeebec4c4a667b658f3c45e8d736b1299888505bb70 WatchSource:0}: Error finding container 06a10996efb0b022fac0edeebec4c4a667b658f3c45e8d736b1299888505bb70: Status 404 returned error can't find the container with id 06a10996efb0b022fac0edeebec4c4a667b658f3c45e8d736b1299888505bb70 Jul 18 01:49:45.427201 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:45.427121 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-7crkn" event={"ID":"725cba84-708c-47ff-ab2b-8d54c646ec0e","Type":"ContainerStarted","Data":"06a10996efb0b022fac0edeebec4c4a667b658f3c45e8d736b1299888505bb70"} Jul 18 01:49:50.026853 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.026817 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-847d84cbd9-snwjj"] Jul 18 01:49:50.043339 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.043297 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-847d84cbd9-snwjj"] Jul 18 01:49:50.043517 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.043462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.046239 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.045988 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 01:49:50.046239 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.046108 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 01:49:50.046467 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.046436 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 01:49:50.046736 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.046713 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 01:49:50.046863 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.046735 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 01:49:50.046927 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.046877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wc4pm\"" Jul 18 01:49:50.177776 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.177741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-service-ca\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.177951 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.177784 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-config\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.177951 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.177811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-serving-cert\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.177951 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.177866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtbwv\" (UniqueName: \"kubernetes.io/projected/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-kube-api-access-vtbwv\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.178120 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.177998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-oauth-config\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.178120 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.178052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-oauth-serving-cert\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.278559 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.278465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-config\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.278559 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.278514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-serving-cert\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.278559 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.278540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtbwv\" (UniqueName: \"kubernetes.io/projected/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-kube-api-access-vtbwv\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.278848 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.278601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-oauth-config\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.278848 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.278640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-oauth-serving-cert\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.278848 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.278670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-service-ca\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.279376 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.279340 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-config\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.279527 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.279502 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-service-ca\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.279649 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.279551 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-oauth-serving-cert\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.281308 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.281287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-oauth-config\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.281410 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.281392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-serving-cert\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.286392 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.286371 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtbwv\" (UniqueName: \"kubernetes.io/projected/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-kube-api-access-vtbwv\") pod \"console-847d84cbd9-snwjj\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.355271 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.355235 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:49:50.498620 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:50.498569 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-847d84cbd9-snwjj"] Jul 18 01:49:50.502472 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:49:50.502431 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8406a753_c2a1_45a4_a9d7_036fc03d6bfc.slice/crio-3a06bd23b84d156cf2a8208c308e31bd8b6ca80c4d10c49428f1bbf0c8ebf4f9 WatchSource:0}: Error finding container 3a06bd23b84d156cf2a8208c308e31bd8b6ca80c4d10c49428f1bbf0c8ebf4f9: Status 404 returned error can't find the container with id 3a06bd23b84d156cf2a8208c308e31bd8b6ca80c4d10c49428f1bbf0c8ebf4f9 Jul 18 01:49:51.454031 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:51.453985 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-847d84cbd9-snwjj" event={"ID":"8406a753-c2a1-45a4-a9d7-036fc03d6bfc","Type":"ContainerStarted","Data":"3a06bd23b84d156cf2a8208c308e31bd8b6ca80c4d10c49428f1bbf0c8ebf4f9"} Jul 18 01:49:54.467845 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:54.467803 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-847d84cbd9-snwjj" event={"ID":"8406a753-c2a1-45a4-a9d7-036fc03d6bfc","Type":"ContainerStarted","Data":"49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc"} Jul 18 01:49:54.485022 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:54.484976 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-847d84cbd9-snwjj" podStartSLOduration=1.5393185649999999 podStartE2EDuration="4.48496018s" totalImagesPullingTime="2.945641615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:54.483954865 +0000 UTC m=+100.167827604" watchObservedRunningTime="2026-07-18 01:49:54.48496018 +0000 UTC m=+100.168832918" Jul 18 01:49:57.004962 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.004926 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cc4cfc5d4-d45wg"] Jul 18 01:49:57.083490 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.083455 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc4cfc5d4-d45wg"] Jul 18 01:49:57.083677 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.083566 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.090323 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.090298 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 18 01:49:57.239862 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.239813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-oauth-config\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.239862 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.239858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-console-config\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.240106 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.239887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-oauth-serving-cert\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.240106 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.239932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-serving-cert\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.240106 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.240012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbsxr\" (UniqueName: \"kubernetes.io/projected/2765f425-9839-469d-ae83-4d7de3c57ce4-kube-api-access-zbsxr\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.240106 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.240072 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-trusted-ca-bundle\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.240290 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.240143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-service-ca\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340720 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340634 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-serving-cert\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340720 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbsxr\" (UniqueName: \"kubernetes.io/projected/2765f425-9839-469d-ae83-4d7de3c57ce4-kube-api-access-zbsxr\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340943 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-trusted-ca-bundle\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340943 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-service-ca\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340943 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-oauth-config\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340943 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-console-config\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.340943 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.340900 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-oauth-serving-cert\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.341855 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.341798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-trusted-ca-bundle\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.342074 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.342029 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-service-ca\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.342315 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.342291 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-console-config\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.342389 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.342315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-oauth-serving-cert\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.343518 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.343496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-serving-cert\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.343518 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.343523 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-oauth-config\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.355090 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.355067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbsxr\" (UniqueName: \"kubernetes.io/projected/2765f425-9839-469d-ae83-4d7de3c57ce4-kube-api-access-zbsxr\") pod \"console-cc4cfc5d4-d45wg\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:49:57.394642 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:49:57.394619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:50:00.356335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:00.356291 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:50:00.356902 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:00.356368 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:50:00.361783 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:00.361760 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-frd7c" Jul 18 01:50:00.362995 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:00.362976 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:50:00.495971 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:00.495939 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:50:01.519371 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:01.519223 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cc4cfc5d4-d45wg"] Jul 18 01:50:01.521925 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:50:01.521884 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2765f425_9839_469d_ae83_4d7de3c57ce4.slice/crio-f9ec256532aa25d0e30d3eb81cc6197a92b829743eeff687941adbc5a26490d2 WatchSource:0}: Error finding container f9ec256532aa25d0e30d3eb81cc6197a92b829743eeff687941adbc5a26490d2: Status 404 returned error can't find the container with id f9ec256532aa25d0e30d3eb81cc6197a92b829743eeff687941adbc5a26490d2 Jul 18 01:50:02.502067 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:02.502020 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc4cfc5d4-d45wg" event={"ID":"2765f425-9839-469d-ae83-4d7de3c57ce4","Type":"ContainerStarted","Data":"790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2"} Jul 18 01:50:02.502264 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:02.502121 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc4cfc5d4-d45wg" event={"ID":"2765f425-9839-469d-ae83-4d7de3c57ce4","Type":"ContainerStarted","Data":"f9ec256532aa25d0e30d3eb81cc6197a92b829743eeff687941adbc5a26490d2"} Jul 18 01:50:02.503873 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:02.503830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-7crkn" event={"ID":"725cba84-708c-47ff-ab2b-8d54c646ec0e","Type":"ContainerStarted","Data":"74adac1d65a74e31d3f360de7a6d7766a518b7495933437889740393453bf88d"} Jul 18 01:50:02.504024 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:02.503995 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:50:02.517692 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:02.517661 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-7crkn" Jul 18 01:50:02.519248 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:02.519200 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cc4cfc5d4-d45wg" podStartSLOduration=6.519170079 podStartE2EDuration="6.519170079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:02.518657926 +0000 UTC m=+108.202530661" watchObservedRunningTime="2026-07-18 01:50:02.519170079 +0000 UTC m=+108.203042809" Jul 18 01:50:07.395499 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:07.395458 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:50:07.396065 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:07.395528 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:50:07.401165 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:07.401143 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:50:07.422465 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:07.422420 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-7crkn" podStartSLOduration=6.756020024 podStartE2EDuration="23.422406067s" totalImagesPullingTime="16.666386043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:02.536031629 +0000 UTC m=+108.219904367" watchObservedRunningTime="2026-07-18 01:50:07.422406067 +0000 UTC m=+113.106278869" Jul 18 01:50:07.529526 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:07.529500 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:50:07.573229 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:07.573203 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-847d84cbd9-snwjj"] Jul 18 01:50:32.601920 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.601875 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-847d84cbd9-snwjj" podUID="8406a753-c2a1-45a4-a9d7-036fc03d6bfc" containerName="console" containerID="cri-o://49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc" gracePeriod=15 Jul 18 01:50:32.877502 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.877430 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:50:32.915143 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915116 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-service-ca\") pod \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " Jul 18 01:50:32.915305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915152 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-serving-cert\") pod \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " Jul 18 01:50:32.915305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915201 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtbwv\" (UniqueName: \"kubernetes.io/projected/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-kube-api-access-vtbwv\") pod \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " Jul 18 01:50:32.915305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915225 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-oauth-serving-cert\") pod \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " Jul 18 01:50:32.915305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915247 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-oauth-config\") pod \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " Jul 18 01:50:32.915305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915267 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-config\") pod \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\" (UID: \"8406a753-c2a1-45a4-a9d7-036fc03d6bfc\") " Jul 18 01:50:32.915660 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915626 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-service-ca" (OuterVolumeSpecName: "service-ca") pod "8406a753-c2a1-45a4-a9d7-036fc03d6bfc" (UID: "8406a753-c2a1-45a4-a9d7-036fc03d6bfc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:50:32.915765 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915715 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8406a753-c2a1-45a4-a9d7-036fc03d6bfc" (UID: "8406a753-c2a1-45a4-a9d7-036fc03d6bfc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:50:32.915765 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.915743 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-config" (OuterVolumeSpecName: "console-config") pod "8406a753-c2a1-45a4-a9d7-036fc03d6bfc" (UID: "8406a753-c2a1-45a4-a9d7-036fc03d6bfc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:50:32.917528 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.917506 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-kube-api-access-vtbwv" (OuterVolumeSpecName: "kube-api-access-vtbwv") pod "8406a753-c2a1-45a4-a9d7-036fc03d6bfc" (UID: "8406a753-c2a1-45a4-a9d7-036fc03d6bfc"). InnerVolumeSpecName "kube-api-access-vtbwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:50:32.917528 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.917507 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8406a753-c2a1-45a4-a9d7-036fc03d6bfc" (UID: "8406a753-c2a1-45a4-a9d7-036fc03d6bfc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:50:32.917668 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:32.917516 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8406a753-c2a1-45a4-a9d7-036fc03d6bfc" (UID: "8406a753-c2a1-45a4-a9d7-036fc03d6bfc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:50:33.016050 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.016009 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtbwv\" (UniqueName: \"kubernetes.io/projected/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-kube-api-access-vtbwv\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:50:33.016050 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.016042 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-oauth-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:50:33.016050 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.016052 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-oauth-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:50:33.016050 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.016061 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:50:33.016302 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.016071 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-service-ca\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:50:33.016302 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.016080 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8406a753-c2a1-45a4-a9d7-036fc03d6bfc-console-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:50:33.615881 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.615843 2580 generic.go:356] "Generic (PLEG): container finished" podID="8406a753-c2a1-45a4-a9d7-036fc03d6bfc" containerID="49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc" exitCode=0 Jul 18 01:50:33.616265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.615890 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-847d84cbd9-snwjj" Jul 18 01:50:33.616265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.615923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-847d84cbd9-snwjj" event={"ID":"8406a753-c2a1-45a4-a9d7-036fc03d6bfc","Type":"ContainerDied","Data":"49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc"} Jul 18 01:50:33.616265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.615974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-847d84cbd9-snwjj" event={"ID":"8406a753-c2a1-45a4-a9d7-036fc03d6bfc","Type":"ContainerDied","Data":"3a06bd23b84d156cf2a8208c308e31bd8b6ca80c4d10c49428f1bbf0c8ebf4f9"} Jul 18 01:50:33.616265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.616003 2580 scope.go:117] "RemoveContainer" containerID="49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc" Jul 18 01:50:33.624804 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.624789 2580 scope.go:117] "RemoveContainer" containerID="49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc" Jul 18 01:50:33.625090 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:50:33.625070 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc\": container with ID starting with 49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc not found: ID does not exist" containerID="49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc" Jul 18 01:50:33.625139 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.625098 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc"} err="failed to get container status \"49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc\": rpc error: code = NotFound desc = could not find container \"49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc\": container with ID starting with 49971b4a31783ba403f57de364cc99089176f77f43516908cd4da1c60a2799dc not found: ID does not exist" Jul 18 01:50:33.636334 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.636312 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-847d84cbd9-snwjj"] Jul 18 01:50:33.639139 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:33.639122 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-847d84cbd9-snwjj"] Jul 18 01:50:34.896508 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:50:34.896474 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8406a753-c2a1-45a4-a9d7-036fc03d6bfc" path="/var/lib/kubelet/pods/8406a753-c2a1-45a4-a9d7-036fc03d6bfc/volumes" Jul 18 01:51:04.266716 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.266681 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c74bcc8d4-zl467"] Jul 18 01:51:04.267421 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.266936 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8406a753-c2a1-45a4-a9d7-036fc03d6bfc" containerName="console" Jul 18 01:51:04.267421 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.266947 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8406a753-c2a1-45a4-a9d7-036fc03d6bfc" containerName="console" Jul 18 01:51:04.267421 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.267006 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8406a753-c2a1-45a4-a9d7-036fc03d6bfc" containerName="console" Jul 18 01:51:04.269387 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.269371 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.280000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.279978 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c74bcc8d4-zl467"] Jul 18 01:51:04.339380 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-trusted-ca-bundle\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.339506 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339386 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-oauth-config\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.339506 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-console-config\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.339506 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-service-ca\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.339706 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-oauth-serving-cert\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.339706 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w567\" (UniqueName: \"kubernetes.io/projected/acd544f6-f8e7-4d93-9650-587977d479fb-kube-api-access-4w567\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.339706 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.339645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-serving-cert\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440090 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-serving-cert\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440200 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-trusted-ca-bundle\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440200 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-oauth-config\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-console-config\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-service-ca\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440305 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-oauth-serving-cert\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440472 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w567\" (UniqueName: \"kubernetes.io/projected/acd544f6-f8e7-4d93-9650-587977d479fb-kube-api-access-4w567\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440978 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440954 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-console-config\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.440978 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440970 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-trusted-ca-bundle\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.441133 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-oauth-serving-cert\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.441133 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.440987 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-service-ca\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.442614 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.442567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-serving-cert\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.442690 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.442616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-oauth-config\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.448560 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.448533 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w567\" (UniqueName: \"kubernetes.io/projected/acd544f6-f8e7-4d93-9650-587977d479fb-kube-api-access-4w567\") pod \"console-6c74bcc8d4-zl467\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.577761 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.577710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:04.717805 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:04.717773 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c74bcc8d4-zl467"] Jul 18 01:51:04.720244 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:51:04.720218 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd544f6_f8e7_4d93_9650_587977d479fb.slice/crio-0db49a73990bc1d1728652fe2b9bae8f19cca759ebd9afcbd643695df1e46512 WatchSource:0}: Error finding container 0db49a73990bc1d1728652fe2b9bae8f19cca759ebd9afcbd643695df1e46512: Status 404 returned error can't find the container with id 0db49a73990bc1d1728652fe2b9bae8f19cca759ebd9afcbd643695df1e46512 Jul 18 01:51:05.728108 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:05.728073 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c74bcc8d4-zl467" event={"ID":"acd544f6-f8e7-4d93-9650-587977d479fb","Type":"ContainerStarted","Data":"1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208"} Jul 18 01:51:05.728472 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:05.728153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c74bcc8d4-zl467" event={"ID":"acd544f6-f8e7-4d93-9650-587977d479fb","Type":"ContainerStarted","Data":"0db49a73990bc1d1728652fe2b9bae8f19cca759ebd9afcbd643695df1e46512"} Jul 18 01:51:05.744307 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:05.744261 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c74bcc8d4-zl467" podStartSLOduration=1.744245367 podStartE2EDuration="1.744245367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:05.743836734 +0000 UTC m=+171.427709475" watchObservedRunningTime="2026-07-18 01:51:05.744245367 +0000 UTC m=+171.428118109" Jul 18 01:51:14.578113 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:14.578028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:14.578113 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:14.578087 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:14.582778 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:14.582757 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:14.761910 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:14.761885 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:51:14.807986 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:14.807959 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc4cfc5d4-d45wg"] Jul 18 01:51:39.830567 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:39.830498 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cc4cfc5d4-d45wg" podUID="2765f425-9839-469d-ae83-4d7de3c57ce4" containerName="console" containerID="cri-o://790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2" gracePeriod=15 Jul 18 01:51:40.071471 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.071443 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:51:40.092411 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092333 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-trusted-ca-bundle\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.092411 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092387 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-oauth-config\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.092630 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092427 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-console-config\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.092630 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092453 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-oauth-serving-cert\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.092630 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092480 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-service-ca\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.092630 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092508 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbsxr\" (UniqueName: \"kubernetes.io/projected/2765f425-9839-469d-ae83-4d7de3c57ce4-kube-api-access-zbsxr\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.092630 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-serving-cert\") pod \"2765f425-9839-469d-ae83-4d7de3c57ce4\" (UID: \"2765f425-9839-469d-ae83-4d7de3c57ce4\") " Jul 18 01:51:40.093436 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.093168 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:40.093436 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.093204 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-console-config" (OuterVolumeSpecName: "console-config") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:40.093436 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.093251 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-service-ca" (OuterVolumeSpecName: "service-ca") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:40.093931 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.092785 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:40.095076 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.095049 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:51:40.095234 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.095210 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:51:40.095451 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.095419 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2765f425-9839-469d-ae83-4d7de3c57ce4-kube-api-access-zbsxr" (OuterVolumeSpecName: "kube-api-access-zbsxr") pod "2765f425-9839-469d-ae83-4d7de3c57ce4" (UID: "2765f425-9839-469d-ae83-4d7de3c57ce4"). InnerVolumeSpecName "kube-api-access-zbsxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:51:40.193377 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193337 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-trusted-ca-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.193377 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193373 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-oauth-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.193377 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193383 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-console-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.193645 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193393 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-oauth-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.193645 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193401 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2765f425-9839-469d-ae83-4d7de3c57ce4-service-ca\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.193645 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193409 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbsxr\" (UniqueName: \"kubernetes.io/projected/2765f425-9839-469d-ae83-4d7de3c57ce4-kube-api-access-zbsxr\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.193645 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.193418 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2765f425-9839-469d-ae83-4d7de3c57ce4-console-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:51:40.861131 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.861089 2580 generic.go:356] "Generic (PLEG): container finished" podID="2765f425-9839-469d-ae83-4d7de3c57ce4" containerID="790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2" exitCode=0 Jul 18 01:51:40.861593 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.861146 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cc4cfc5d4-d45wg" Jul 18 01:51:40.861593 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.861177 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc4cfc5d4-d45wg" event={"ID":"2765f425-9839-469d-ae83-4d7de3c57ce4","Type":"ContainerDied","Data":"790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2"} Jul 18 01:51:40.861593 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.861267 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cc4cfc5d4-d45wg" event={"ID":"2765f425-9839-469d-ae83-4d7de3c57ce4","Type":"ContainerDied","Data":"f9ec256532aa25d0e30d3eb81cc6197a92b829743eeff687941adbc5a26490d2"} Jul 18 01:51:40.861593 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.861298 2580 scope.go:117] "RemoveContainer" containerID="790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2" Jul 18 01:51:40.869237 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.869220 2580 scope.go:117] "RemoveContainer" containerID="790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2" Jul 18 01:51:40.869514 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:51:40.869495 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2\": container with ID starting with 790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2 not found: ID does not exist" containerID="790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2" Jul 18 01:51:40.869608 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.869521 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2"} err="failed to get container status \"790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2\": rpc error: code = NotFound desc = could not find container \"790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2\": container with ID starting with 790362f63ce4b35359cfea1fcb8220c902034ab2951ed9e4b1c4bc84d449cda2 not found: ID does not exist" Jul 18 01:51:40.879781 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.879760 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cc4cfc5d4-d45wg"] Jul 18 01:51:40.885372 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.885355 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cc4cfc5d4-d45wg"] Jul 18 01:51:40.895695 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:51:40.895673 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2765f425-9839-469d-ae83-4d7de3c57ce4" path="/var/lib/kubelet/pods/2765f425-9839-469d-ae83-4d7de3c57ce4/volumes" Jul 18 01:52:16.672003 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.671971 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f887f797d-7lhxd"] Jul 18 01:52:16.672401 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.672231 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2765f425-9839-469d-ae83-4d7de3c57ce4" containerName="console" Jul 18 01:52:16.672401 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.672243 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2765f425-9839-469d-ae83-4d7de3c57ce4" containerName="console" Jul 18 01:52:16.672401 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.672291 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2765f425-9839-469d-ae83-4d7de3c57ce4" containerName="console" Jul 18 01:52:16.674592 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.674561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.687624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.687600 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f887f797d-7lhxd"] Jul 18 01:52:16.756359 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-serving-cert\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.756359 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-service-ca\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.756557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756386 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-oauth-config\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.756557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-oauth-serving-cert\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.756557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-config\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.756557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-trusted-ca-bundle\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.756719 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.756562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc87s\" (UniqueName: \"kubernetes.io/projected/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-kube-api-access-cc87s\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.857691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-service-ca\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.857691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-oauth-config\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.857945 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-oauth-serving-cert\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.857945 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857745 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-config\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.857945 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-trusted-ca-bundle\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.857945 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc87s\" (UniqueName: \"kubernetes.io/projected/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-kube-api-access-cc87s\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.858147 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.857935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-serving-cert\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.858438 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.858408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-service-ca\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.858602 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.858506 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-oauth-serving-cert\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.858753 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.858727 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-config\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.858819 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.858782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-trusted-ca-bundle\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.860216 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.860188 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-oauth-config\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.860423 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.860401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-serving-cert\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.865314 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.865297 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc87s\" (UniqueName: \"kubernetes.io/projected/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-kube-api-access-cc87s\") pod \"console-7f887f797d-7lhxd\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:16.983338 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:16.983312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:17.109204 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:17.109177 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f887f797d-7lhxd"] Jul 18 01:52:17.111624 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:52:17.111569 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8cd3c46_9a6d_4371_adc3_d0e65dee6826.slice/crio-730a80b48e04efafcefabb142cbb0233a9ce64430cd8f138238a219b1e9b1a45 WatchSource:0}: Error finding container 730a80b48e04efafcefabb142cbb0233a9ce64430cd8f138238a219b1e9b1a45: Status 404 returned error can't find the container with id 730a80b48e04efafcefabb142cbb0233a9ce64430cd8f138238a219b1e9b1a45 Jul 18 01:52:17.985352 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:17.985315 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f887f797d-7lhxd" event={"ID":"b8cd3c46-9a6d-4371-adc3-d0e65dee6826","Type":"ContainerStarted","Data":"1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0"} Jul 18 01:52:17.985764 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:17.985394 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f887f797d-7lhxd" event={"ID":"b8cd3c46-9a6d-4371-adc3-d0e65dee6826","Type":"ContainerStarted","Data":"730a80b48e04efafcefabb142cbb0233a9ce64430cd8f138238a219b1e9b1a45"} Jul 18 01:52:18.005343 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:18.005290 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f887f797d-7lhxd" podStartSLOduration=2.005274107 podStartE2EDuration="2.005274107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:18.003884964 +0000 UTC m=+243.687757704" watchObservedRunningTime="2026-07-18 01:52:18.005274107 +0000 UTC m=+243.689146845" Jul 18 01:52:26.983675 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:26.983620 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:26.983675 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:26.983689 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:26.988405 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:26.988384 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:27.020202 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:27.020179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:52:27.064612 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:27.064571 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c74bcc8d4-zl467"] Jul 18 01:52:48.007276 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.007194 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n"] Jul 18 01:52:48.010032 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.010015 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.012263 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.012243 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 01:52:48.013196 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.013177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 18 01:52:48.013283 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.013217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 01:52:48.013283 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.013234 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 01:52:48.013283 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.013217 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-tsmwj\"" Jul 18 01:52:48.019908 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.019876 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n"] Jul 18 01:52:48.161353 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.161319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmzjw\" (UniqueName: \"kubernetes.io/projected/4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0-kube-api-access-dmzjw\") pod \"managed-serviceaccount-addon-agent-69564cc8bd-5rw9n\" (UID: \"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.161482 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.161365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69564cc8bd-5rw9n\" (UID: \"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.220755 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.220708 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm"] Jul 18 01:52:48.222768 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.222754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.224886 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.224870 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 18 01:52:48.233310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.233289 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm"] Jul 18 01:52:48.261922 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.261874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69564cc8bd-5rw9n\" (UID: \"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.262018 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.261933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmzjw\" (UniqueName: \"kubernetes.io/projected/4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0-kube-api-access-dmzjw\") pod \"managed-serviceaccount-addon-agent-69564cc8bd-5rw9n\" (UID: \"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.264248 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.264230 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69564cc8bd-5rw9n\" (UID: \"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.269403 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.269381 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmzjw\" (UniqueName: \"kubernetes.io/projected/4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0-kube-api-access-dmzjw\") pod \"managed-serviceaccount-addon-agent-69564cc8bd-5rw9n\" (UID: \"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.327275 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.327253 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" Jul 18 01:52:48.362942 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.362906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjsd6\" (UniqueName: \"kubernetes.io/projected/1d6695dc-8265-4579-97f5-cd4eb07acd47-kube-api-access-xjsd6\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.363068 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.362973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d6695dc-8265-4579-97f5-cd4eb07acd47-tmp\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.363068 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.363007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1d6695dc-8265-4579-97f5-cd4eb07acd47-klusterlet-config\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.447005 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.446982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n"] Jul 18 01:52:48.449468 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:52:48.449443 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e0f94e4_9fc2_48c8_a05f_e7742c48b6c0.slice/crio-442c82550f28c3375e5100da5e7e1d625bbb01260543267c4a04008145d2e3f5 WatchSource:0}: Error finding container 442c82550f28c3375e5100da5e7e1d625bbb01260543267c4a04008145d2e3f5: Status 404 returned error can't find the container with id 442c82550f28c3375e5100da5e7e1d625bbb01260543267c4a04008145d2e3f5 Jul 18 01:52:48.463352 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.463333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjsd6\" (UniqueName: \"kubernetes.io/projected/1d6695dc-8265-4579-97f5-cd4eb07acd47-kube-api-access-xjsd6\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.463452 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.463372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d6695dc-8265-4579-97f5-cd4eb07acd47-tmp\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.463452 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.463396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1d6695dc-8265-4579-97f5-cd4eb07acd47-klusterlet-config\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.463731 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.463706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1d6695dc-8265-4579-97f5-cd4eb07acd47-tmp\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.465762 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.465741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/1d6695dc-8265-4579-97f5-cd4eb07acd47-klusterlet-config\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.471282 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.471261 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjsd6\" (UniqueName: \"kubernetes.io/projected/1d6695dc-8265-4579-97f5-cd4eb07acd47-kube-api-access-xjsd6\") pod \"klusterlet-addon-workmgr-66795c6555-5v8jm\" (UID: \"1d6695dc-8265-4579-97f5-cd4eb07acd47\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.532009 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.531959 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:48.649513 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:48.649483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm"] Jul 18 01:52:48.652078 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:52:48.652039 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6695dc_8265_4579_97f5_cd4eb07acd47.slice/crio-85b5fb1347d5939ec47779d7a3a7f9e24ddcb282be86c60b33232652ff05d21b WatchSource:0}: Error finding container 85b5fb1347d5939ec47779d7a3a7f9e24ddcb282be86c60b33232652ff05d21b: Status 404 returned error can't find the container with id 85b5fb1347d5939ec47779d7a3a7f9e24ddcb282be86c60b33232652ff05d21b Jul 18 01:52:49.091501 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:49.091457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" event={"ID":"1d6695dc-8265-4579-97f5-cd4eb07acd47","Type":"ContainerStarted","Data":"85b5fb1347d5939ec47779d7a3a7f9e24ddcb282be86c60b33232652ff05d21b"} Jul 18 01:52:49.092928 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:49.092899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" event={"ID":"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0","Type":"ContainerStarted","Data":"442c82550f28c3375e5100da5e7e1d625bbb01260543267c4a04008145d2e3f5"} Jul 18 01:52:51.063725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.063403 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2"] Jul 18 01:52:51.067156 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.067117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.069632 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.069598 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qtctr\"" Jul 18 01:52:51.069777 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.069610 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 01:52:51.070420 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.070367 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 01:52:51.072963 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.072944 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2"] Jul 18 01:52:51.187137 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.187113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54mn4\" (UniqueName: \"kubernetes.io/projected/775d43ca-7ed2-4f22-99d6-1e2343220cbb-kube-api-access-54mn4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.187241 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.187162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.187241 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.187227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.287981 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.287889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54mn4\" (UniqueName: \"kubernetes.io/projected/775d43ca-7ed2-4f22-99d6-1e2343220cbb-kube-api-access-54mn4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.287981 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.287957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.288219 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.288033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.288427 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.288402 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.288515 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.288447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.296379 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.296358 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54mn4\" (UniqueName: \"kubernetes.io/projected/775d43ca-7ed2-4f22-99d6-1e2343220cbb-kube-api-access-54mn4\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:51.380194 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:51.380127 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:52:52.084350 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.084304 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c74bcc8d4-zl467" podUID="acd544f6-f8e7-4d93-9650-587977d479fb" containerName="console" containerID="cri-o://1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208" gracePeriod=15 Jul 18 01:52:52.108446 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.108413 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" event={"ID":"4e0f94e4-9fc2-48c8-a05f-e7742c48b6c0","Type":"ContainerStarted","Data":"81571c41adc50088d9ec5e83bf257497d8905142c9803ad2338dad8ff1b815d2"} Jul 18 01:52:52.135783 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.135733 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69564cc8bd-5rw9n" podStartSLOduration=2.470684706 podStartE2EDuration="5.135712399s" totalImagesPullingTime="2.665027693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:52.134890614 +0000 UTC m=+277.818763353" watchObservedRunningTime="2026-07-18 01:52:52.135712399 +0000 UTC m=+277.819585168" Jul 18 01:52:52.521706 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.521681 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2"] Jul 18 01:52:52.524214 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.524193 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:52:52.524619 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:52:52.524562 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775d43ca_7ed2_4f22_99d6_1e2343220cbb.slice/crio-6260aff1b2154383838181715d2eeb434b01c49a7c3ba008d77a1d5af9385b7d WatchSource:0}: Error finding container 6260aff1b2154383838181715d2eeb434b01c49a7c3ba008d77a1d5af9385b7d: Status 404 returned error can't find the container with id 6260aff1b2154383838181715d2eeb434b01c49a7c3ba008d77a1d5af9385b7d Jul 18 01:52:52.703197 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703125 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-oauth-serving-cert\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703197 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703182 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-console-config\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703399 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703208 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-oauth-config\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703399 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703230 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-service-ca\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703399 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703266 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-trusted-ca-bundle\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703399 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703286 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-serving-cert\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703399 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703314 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4w567\" (UniqueName: \"kubernetes.io/projected/acd544f6-f8e7-4d93-9650-587977d479fb-kube-api-access-4w567\") pod \"acd544f6-f8e7-4d93-9650-587977d479fb\" (UID: \"acd544f6-f8e7-4d93-9650-587977d479fb\") " Jul 18 01:52:52.703681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703622 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:52:52.703681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703653 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-service-ca" (OuterVolumeSpecName: "service-ca") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:52:52.703681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703669 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-console-config" (OuterVolumeSpecName: "console-config") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:52:52.703810 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.703781 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:52:52.705329 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.705307 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:52:52.705405 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.705334 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:52:52.705405 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.705342 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd544f6-f8e7-4d93-9650-587977d479fb-kube-api-access-4w567" (OuterVolumeSpecName: "kube-api-access-4w567") pod "acd544f6-f8e7-4d93-9650-587977d479fb" (UID: "acd544f6-f8e7-4d93-9650-587977d479fb"). InnerVolumeSpecName "kube-api-access-4w567". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:52:52.803848 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803820 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:52.803848 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803843 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4w567\" (UniqueName: \"kubernetes.io/projected/acd544f6-f8e7-4d93-9650-587977d479fb-kube-api-access-4w567\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:52.804022 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803854 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-oauth-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:52.804022 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803863 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-console-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:52.804022 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803873 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd544f6-f8e7-4d93-9650-587977d479fb-console-oauth-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:52.804022 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803881 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-service-ca\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:52.804022 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:52.803889 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd544f6-f8e7-4d93-9650-587977d479fb-trusted-ca-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:52:53.114529 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.114488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" event={"ID":"775d43ca-7ed2-4f22-99d6-1e2343220cbb","Type":"ContainerStarted","Data":"6260aff1b2154383838181715d2eeb434b01c49a7c3ba008d77a1d5af9385b7d"} Jul 18 01:52:53.116310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.116273 2580 generic.go:356] "Generic (PLEG): container finished" podID="acd544f6-f8e7-4d93-9650-587977d479fb" containerID="1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208" exitCode=0 Jul 18 01:52:53.116501 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.116332 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c74bcc8d4-zl467" Jul 18 01:52:53.116501 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.116373 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c74bcc8d4-zl467" event={"ID":"acd544f6-f8e7-4d93-9650-587977d479fb","Type":"ContainerDied","Data":"1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208"} Jul 18 01:52:53.116501 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.116410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c74bcc8d4-zl467" event={"ID":"acd544f6-f8e7-4d93-9650-587977d479fb","Type":"ContainerDied","Data":"0db49a73990bc1d1728652fe2b9bae8f19cca759ebd9afcbd643695df1e46512"} Jul 18 01:52:53.116501 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.116436 2580 scope.go:117] "RemoveContainer" containerID="1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208" Jul 18 01:52:53.119996 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.119957 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" event={"ID":"1d6695dc-8265-4579-97f5-cd4eb07acd47","Type":"ContainerStarted","Data":"2ae2db7039796e57d8a0f156815e106b6a38e0e9187e106bccc9aef855294074"} Jul 18 01:52:53.120135 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.120011 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:53.121887 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.121868 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" Jul 18 01:52:53.127342 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.127325 2580 scope.go:117] "RemoveContainer" containerID="1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208" Jul 18 01:52:53.127628 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:52:53.127606 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208\": container with ID starting with 1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208 not found: ID does not exist" containerID="1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208" Jul 18 01:52:53.127712 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.127637 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208"} err="failed to get container status \"1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208\": rpc error: code = NotFound desc = could not find container \"1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208\": container with ID starting with 1eefbfe99336c05fa31ad45522c15fcf6bdb73632289186e7e8b3bd102131208 not found: ID does not exist" Jul 18 01:52:53.144052 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.144007 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66795c6555-5v8jm" podStartSLOduration=1.367110585 podStartE2EDuration="5.143990148s" totalImagesPullingTime="3.776879563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:53.14334461 +0000 UTC m=+278.827217349" watchObservedRunningTime="2026-07-18 01:52:53.143990148 +0000 UTC m=+278.827862888" Jul 18 01:52:53.156265 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.156243 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c74bcc8d4-zl467"] Jul 18 01:52:53.161033 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:53.161010 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c74bcc8d4-zl467"] Jul 18 01:52:54.897131 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:54.897098 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd544f6-f8e7-4d93-9650-587977d479fb" path="/var/lib/kubelet/pods/acd544f6-f8e7-4d93-9650-587977d479fb/volumes" Jul 18 01:52:59.147097 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:59.147061 2580 generic.go:356] "Generic (PLEG): container finished" podID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerID="adea2b3e21e2a197df7f024770591dcb119af7be0d0a56d5c7e8a8036f9400ef" exitCode=0 Jul 18 01:52:59.147498 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:52:59.147115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" event={"ID":"775d43ca-7ed2-4f22-99d6-1e2343220cbb","Type":"ContainerDied","Data":"adea2b3e21e2a197df7f024770591dcb119af7be0d0a56d5c7e8a8036f9400ef"} Jul 18 01:53:02.159534 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:02.159499 2580 generic.go:356] "Generic (PLEG): container finished" podID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerID="7e1c19ccacc94347be45c4f861cba177f6beeee246711066f60a8b5ce2c195ce" exitCode=0 Jul 18 01:53:02.159937 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:02.159569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" event={"ID":"775d43ca-7ed2-4f22-99d6-1e2343220cbb","Type":"ContainerDied","Data":"7e1c19ccacc94347be45c4f861cba177f6beeee246711066f60a8b5ce2c195ce"} Jul 18 01:53:10.193916 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:10.193878 2580 generic.go:356] "Generic (PLEG): container finished" podID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerID="4e5d178425e7465205c14cd292786cc99777cf78b15ff7b0c52119064d9b6d8d" exitCode=0 Jul 18 01:53:10.193916 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:10.193919 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" event={"ID":"775d43ca-7ed2-4f22-99d6-1e2343220cbb","Type":"ContainerDied","Data":"4e5d178425e7465205c14cd292786cc99777cf78b15ff7b0c52119064d9b6d8d"} Jul 18 01:53:11.329050 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.329025 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:53:11.439477 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.439440 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-util\") pod \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " Jul 18 01:53:11.439667 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.439492 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54mn4\" (UniqueName: \"kubernetes.io/projected/775d43ca-7ed2-4f22-99d6-1e2343220cbb-kube-api-access-54mn4\") pod \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " Jul 18 01:53:11.439667 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.439534 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-bundle\") pod \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\" (UID: \"775d43ca-7ed2-4f22-99d6-1e2343220cbb\") " Jul 18 01:53:11.440240 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.440212 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-bundle" (OuterVolumeSpecName: "bundle") pod "775d43ca-7ed2-4f22-99d6-1e2343220cbb" (UID: "775d43ca-7ed2-4f22-99d6-1e2343220cbb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:11.441728 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.441698 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775d43ca-7ed2-4f22-99d6-1e2343220cbb-kube-api-access-54mn4" (OuterVolumeSpecName: "kube-api-access-54mn4") pod "775d43ca-7ed2-4f22-99d6-1e2343220cbb" (UID: "775d43ca-7ed2-4f22-99d6-1e2343220cbb"). InnerVolumeSpecName "kube-api-access-54mn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:53:11.443982 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.443957 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-util" (OuterVolumeSpecName: "util") pod "775d43ca-7ed2-4f22-99d6-1e2343220cbb" (UID: "775d43ca-7ed2-4f22-99d6-1e2343220cbb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:11.540335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.540303 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:11.540335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.540333 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54mn4\" (UniqueName: \"kubernetes.io/projected/775d43ca-7ed2-4f22-99d6-1e2343220cbb-kube-api-access-54mn4\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:11.540335 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:11.540343 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/775d43ca-7ed2-4f22-99d6-1e2343220cbb-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:12.202674 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:12.202636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" event={"ID":"775d43ca-7ed2-4f22-99d6-1e2343220cbb","Type":"ContainerDied","Data":"6260aff1b2154383838181715d2eeb434b01c49a7c3ba008d77a1d5af9385b7d"} Jul 18 01:53:12.202674 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:12.202655 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sgbd2" Jul 18 01:53:12.202867 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:12.202701 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6260aff1b2154383838181715d2eeb434b01c49a7c3ba008d77a1d5af9385b7d" Jul 18 01:53:14.730809 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:14.730783 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:53:14.733878 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:14.733853 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 01:53:14.737483 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:14.737464 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:53:19.353194 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353161 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh"] Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353427 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="extract" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353437 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="extract" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353447 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd544f6-f8e7-4d93-9650-587977d479fb" containerName="console" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353452 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd544f6-f8e7-4d93-9650-587977d479fb" containerName="console" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353460 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="util" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353465 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="util" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353473 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="pull" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353478 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="pull" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353521 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="775d43ca-7ed2-4f22-99d6-1e2343220cbb" containerName="extract" Jul 18 01:53:19.354681 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.353528 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd544f6-f8e7-4d93-9650-587977d479fb" containerName="console" Jul 18 01:53:19.355123 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.355108 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.358257 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.358237 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 18 01:53:19.358853 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.358836 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:53:19.359162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.359146 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xp6lp\"" Jul 18 01:53:19.370957 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.370938 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh"] Jul 18 01:53:19.398885 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.398857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a5f09f-af8d-4371-abab-76f44730f25d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5v5jh\" (UID: \"68a5f09f-af8d-4371-abab-76f44730f25d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.398998 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.398904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbqw\" (UniqueName: \"kubernetes.io/projected/68a5f09f-af8d-4371-abab-76f44730f25d-kube-api-access-bgbqw\") pod \"cert-manager-operator-controller-manager-55c444bf86-5v5jh\" (UID: \"68a5f09f-af8d-4371-abab-76f44730f25d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.500103 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.500073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a5f09f-af8d-4371-abab-76f44730f25d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5v5jh\" (UID: \"68a5f09f-af8d-4371-abab-76f44730f25d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.500236 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.500128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbqw\" (UniqueName: \"kubernetes.io/projected/68a5f09f-af8d-4371-abab-76f44730f25d-kube-api-access-bgbqw\") pod \"cert-manager-operator-controller-manager-55c444bf86-5v5jh\" (UID: \"68a5f09f-af8d-4371-abab-76f44730f25d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.500460 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.500438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a5f09f-af8d-4371-abab-76f44730f25d-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5v5jh\" (UID: \"68a5f09f-af8d-4371-abab-76f44730f25d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.508474 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.508445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbqw\" (UniqueName: \"kubernetes.io/projected/68a5f09f-af8d-4371-abab-76f44730f25d-kube-api-access-bgbqw\") pod \"cert-manager-operator-controller-manager-55c444bf86-5v5jh\" (UID: \"68a5f09f-af8d-4371-abab-76f44730f25d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.663939 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.663853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" Jul 18 01:53:19.795342 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.795289 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh"] Jul 18 01:53:19.798221 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:53:19.798194 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a5f09f_af8d_4371_abab_76f44730f25d.slice/crio-11f0cd260e527b25e1b300746d1473ce5e4438f360375737c6c5ce919f1fe858 WatchSource:0}: Error finding container 11f0cd260e527b25e1b300746d1473ce5e4438f360375737c6c5ce919f1fe858: Status 404 returned error can't find the container with id 11f0cd260e527b25e1b300746d1473ce5e4438f360375737c6c5ce919f1fe858 Jul 18 01:53:19.800711 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:19.800693 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:53:20.232478 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:20.232437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" event={"ID":"68a5f09f-af8d-4371-abab-76f44730f25d","Type":"ContainerStarted","Data":"11f0cd260e527b25e1b300746d1473ce5e4438f360375737c6c5ce919f1fe858"} Jul 18 01:53:22.243657 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:22.243616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" event={"ID":"68a5f09f-af8d-4371-abab-76f44730f25d","Type":"ContainerStarted","Data":"6f43dd2c648ce3287ffb0de4588ad84f98252e274a1a2380c199cafcbe8d01f9"} Jul 18 01:53:22.274820 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:22.271405 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5v5jh" podStartSLOduration=1.43715593 podStartE2EDuration="3.271385423s" totalImagesPullingTime="1.834229493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:22.269290718 +0000 UTC m=+307.953163455" watchObservedRunningTime="2026-07-18 01:53:22.271385423 +0000 UTC m=+307.955258162" Jul 18 01:53:24.009889 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.009855 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z"] Jul 18 01:53:24.012694 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.012676 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.014940 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.014918 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 01:53:24.015045 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.014918 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 01:53:24.015541 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.015527 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qtctr\"" Jul 18 01:53:24.021529 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.021502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z"] Jul 18 01:53:24.140076 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.140027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.140254 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.140082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8bdj\" (UniqueName: \"kubernetes.io/projected/77aa945c-8701-4da8-a74e-5fd437b94c6b-kube-api-access-d8bdj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.140254 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.140141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.240705 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.240660 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.240860 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.240715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8bdj\" (UniqueName: \"kubernetes.io/projected/77aa945c-8701-4da8-a74e-5fd437b94c6b-kube-api-access-d8bdj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.240860 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.240769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.241149 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.241128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.241187 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.241138 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.253346 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.253318 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8bdj\" (UniqueName: \"kubernetes.io/projected/77aa945c-8701-4da8-a74e-5fd437b94c6b-kube-api-access-d8bdj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.322381 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.322290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:24.457380 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:24.457339 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z"] Jul 18 01:53:24.460415 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:53:24.460378 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77aa945c_8701_4da8_a74e_5fd437b94c6b.slice/crio-953f9e8547ab6fc747bc0aa5023644a4e7440393df5a40487cc14e00435be6de WatchSource:0}: Error finding container 953f9e8547ab6fc747bc0aa5023644a4e7440393df5a40487cc14e00435be6de: Status 404 returned error can't find the container with id 953f9e8547ab6fc747bc0aa5023644a4e7440393df5a40487cc14e00435be6de Jul 18 01:53:25.258415 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:25.258382 2580 generic.go:356] "Generic (PLEG): container finished" podID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerID="2ed2b5f7805e73a1bb9a70a8d40253e72a7fecf9144f47cf1b852cfcb773d353" exitCode=0 Jul 18 01:53:25.258761 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:25.258488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" event={"ID":"77aa945c-8701-4da8-a74e-5fd437b94c6b","Type":"ContainerDied","Data":"2ed2b5f7805e73a1bb9a70a8d40253e72a7fecf9144f47cf1b852cfcb773d353"} Jul 18 01:53:25.258761 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:25.258537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" event={"ID":"77aa945c-8701-4da8-a74e-5fd437b94c6b","Type":"ContainerStarted","Data":"953f9e8547ab6fc747bc0aa5023644a4e7440393df5a40487cc14e00435be6de"} Jul 18 01:53:27.269561 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:27.269525 2580 generic.go:356] "Generic (PLEG): container finished" podID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerID="10999fd65d1faf38b6a474daca89e3c731158cb93659c9c4af57493c26dbfd3c" exitCode=0 Jul 18 01:53:27.269977 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:27.269615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" event={"ID":"77aa945c-8701-4da8-a74e-5fd437b94c6b","Type":"ContainerDied","Data":"10999fd65d1faf38b6a474daca89e3c731158cb93659c9c4af57493c26dbfd3c"} Jul 18 01:53:28.275643 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:28.275605 2580 generic.go:356] "Generic (PLEG): container finished" podID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerID="9701169bbb40962e7a92c31e64ad492675f4290499eb18df44cbc0b37b500770" exitCode=0 Jul 18 01:53:28.276025 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:28.275661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" event={"ID":"77aa945c-8701-4da8-a74e-5fd437b94c6b","Type":"ContainerDied","Data":"9701169bbb40962e7a92c31e64ad492675f4290499eb18df44cbc0b37b500770"} Jul 18 01:53:29.411560 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.411532 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:29.485545 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.485507 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-bundle\") pod \"77aa945c-8701-4da8-a74e-5fd437b94c6b\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " Jul 18 01:53:29.485738 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.485570 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-util\") pod \"77aa945c-8701-4da8-a74e-5fd437b94c6b\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " Jul 18 01:53:29.485738 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.485634 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8bdj\" (UniqueName: \"kubernetes.io/projected/77aa945c-8701-4da8-a74e-5fd437b94c6b-kube-api-access-d8bdj\") pod \"77aa945c-8701-4da8-a74e-5fd437b94c6b\" (UID: \"77aa945c-8701-4da8-a74e-5fd437b94c6b\") " Jul 18 01:53:29.485947 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.485923 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-bundle" (OuterVolumeSpecName: "bundle") pod "77aa945c-8701-4da8-a74e-5fd437b94c6b" (UID: "77aa945c-8701-4da8-a74e-5fd437b94c6b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:29.487938 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.487894 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77aa945c-8701-4da8-a74e-5fd437b94c6b-kube-api-access-d8bdj" (OuterVolumeSpecName: "kube-api-access-d8bdj") pod "77aa945c-8701-4da8-a74e-5fd437b94c6b" (UID: "77aa945c-8701-4da8-a74e-5fd437b94c6b"). InnerVolumeSpecName "kube-api-access-d8bdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:53:29.491216 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.491184 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-util" (OuterVolumeSpecName: "util") pod "77aa945c-8701-4da8-a74e-5fd437b94c6b" (UID: "77aa945c-8701-4da8-a74e-5fd437b94c6b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:29.586355 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.586277 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:29.586355 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.586307 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8bdj\" (UniqueName: \"kubernetes.io/projected/77aa945c-8701-4da8-a74e-5fd437b94c6b-kube-api-access-d8bdj\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:29.586355 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:29.586319 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77aa945c-8701-4da8-a74e-5fd437b94c6b-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:30.286140 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:30.286113 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" Jul 18 01:53:30.286325 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:30.286116 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmfl7z" event={"ID":"77aa945c-8701-4da8-a74e-5fd437b94c6b","Type":"ContainerDied","Data":"953f9e8547ab6fc747bc0aa5023644a4e7440393df5a40487cc14e00435be6de"} Jul 18 01:53:30.286325 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:30.286255 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="953f9e8547ab6fc747bc0aa5023644a4e7440393df5a40487cc14e00435be6de" Jul 18 01:53:36.663374 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663334 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl"] Jul 18 01:53:36.663872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663737 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="extract" Jul 18 01:53:36.663872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663755 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="extract" Jul 18 01:53:36.663872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663777 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="pull" Jul 18 01:53:36.663872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663784 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="pull" Jul 18 01:53:36.663872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663806 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="util" Jul 18 01:53:36.663872 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663813 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="util" Jul 18 01:53:36.664163 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.663877 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77aa945c-8701-4da8-a74e-5fd437b94c6b" containerName="extract" Jul 18 01:53:36.665904 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.665885 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.667830 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.667808 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 18 01:53:36.668568 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.668551 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:53:36.668690 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.668665 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-ndxvn\"" Jul 18 01:53:36.675754 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.675732 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl"] Jul 18 01:53:36.747195 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.747158 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t2vg\" (UniqueName: \"kubernetes.io/projected/dba61589-c1ca-4ddb-aa83-1520f5a50519-kube-api-access-5t2vg\") pod \"openshift-lws-operator-fd8ccff4c-xrmvl\" (UID: \"dba61589-c1ca-4ddb-aa83-1520f5a50519\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.747365 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.747210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba61589-c1ca-4ddb-aa83-1520f5a50519-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xrmvl\" (UID: \"dba61589-c1ca-4ddb-aa83-1520f5a50519\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.848615 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.848558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t2vg\" (UniqueName: \"kubernetes.io/projected/dba61589-c1ca-4ddb-aa83-1520f5a50519-kube-api-access-5t2vg\") pod \"openshift-lws-operator-fd8ccff4c-xrmvl\" (UID: \"dba61589-c1ca-4ddb-aa83-1520f5a50519\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.848770 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.848623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba61589-c1ca-4ddb-aa83-1520f5a50519-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xrmvl\" (UID: \"dba61589-c1ca-4ddb-aa83-1520f5a50519\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.848966 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.848945 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dba61589-c1ca-4ddb-aa83-1520f5a50519-tmp\") pod \"openshift-lws-operator-fd8ccff4c-xrmvl\" (UID: \"dba61589-c1ca-4ddb-aa83-1520f5a50519\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.856363 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.856343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t2vg\" (UniqueName: \"kubernetes.io/projected/dba61589-c1ca-4ddb-aa83-1520f5a50519-kube-api-access-5t2vg\") pod \"openshift-lws-operator-fd8ccff4c-xrmvl\" (UID: \"dba61589-c1ca-4ddb-aa83-1520f5a50519\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:36.974628 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:36.974601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" Jul 18 01:53:37.100190 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:37.100158 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl"] Jul 18 01:53:37.103313 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:53:37.103280 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddba61589_c1ca_4ddb_aa83_1520f5a50519.slice/crio-23262e2a44a245bea4ae8b94c7c7f0ab547515a57f2d80d376c098bcf731ba26 WatchSource:0}: Error finding container 23262e2a44a245bea4ae8b94c7c7f0ab547515a57f2d80d376c098bcf731ba26: Status 404 returned error can't find the container with id 23262e2a44a245bea4ae8b94c7c7f0ab547515a57f2d80d376c098bcf731ba26 Jul 18 01:53:37.316178 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:37.316097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" event={"ID":"dba61589-c1ca-4ddb-aa83-1520f5a50519","Type":"ContainerStarted","Data":"23262e2a44a245bea4ae8b94c7c7f0ab547515a57f2d80d376c098bcf731ba26"} Jul 18 01:53:39.326139 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:39.326101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" event={"ID":"dba61589-c1ca-4ddb-aa83-1520f5a50519","Type":"ContainerStarted","Data":"b5b1220873934432a4ba3c9b07f73c089e59dc748c647d3a02528f7b956040cb"} Jul 18 01:53:39.344345 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:39.344294 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-xrmvl" podStartSLOduration=1.351238342 podStartE2EDuration="3.3442789s" totalImagesPullingTime="1.993040558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:39.343024973 +0000 UTC m=+325.026897711" watchObservedRunningTime="2026-07-18 01:53:39.3442789 +0000 UTC m=+325.028151638" Jul 18 01:53:41.341495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.341463 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8"] Jul 18 01:53:41.363799 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.363761 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8"] Jul 18 01:53:41.363942 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.363878 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.366728 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.366700 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 18 01:53:41.366854 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.366736 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 18 01:53:41.366854 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.366744 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qtctr\"" Jul 18 01:53:41.486052 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.486011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.486052 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.486055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.486296 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.486148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76q8c\" (UniqueName: \"kubernetes.io/projected/ee07fcbc-cc03-45ac-8237-0974a61d5130-kube-api-access-76q8c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.587323 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.587285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76q8c\" (UniqueName: \"kubernetes.io/projected/ee07fcbc-cc03-45ac-8237-0974a61d5130-kube-api-access-76q8c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.587495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.587338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.587495 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.587374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.587762 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.587746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.587803 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.587786 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.595330 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.595267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76q8c\" (UniqueName: \"kubernetes.io/projected/ee07fcbc-cc03-45ac-8237-0974a61d5130-kube-api-access-76q8c\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.673096 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.673066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:41.809504 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:41.809477 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8"] Jul 18 01:53:41.811476 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:53:41.811450 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee07fcbc_cc03_45ac_8237_0974a61d5130.slice/crio-7d0ed41123ccf977ad0bf80b9e150946973d44b455fab8b81f29c9f91c7cd527 WatchSource:0}: Error finding container 7d0ed41123ccf977ad0bf80b9e150946973d44b455fab8b81f29c9f91c7cd527: Status 404 returned error can't find the container with id 7d0ed41123ccf977ad0bf80b9e150946973d44b455fab8b81f29c9f91c7cd527 Jul 18 01:53:42.338633 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:42.338599 2580 generic.go:356] "Generic (PLEG): container finished" podID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerID="dbf5bbb373c2cb0e35857f69dfed47ed0ebfafbb955139b2b3d7942a74ec6a56" exitCode=0 Jul 18 01:53:42.338802 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:42.338650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" event={"ID":"ee07fcbc-cc03-45ac-8237-0974a61d5130","Type":"ContainerDied","Data":"dbf5bbb373c2cb0e35857f69dfed47ed0ebfafbb955139b2b3d7942a74ec6a56"} Jul 18 01:53:42.338802 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:42.338684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" event={"ID":"ee07fcbc-cc03-45ac-8237-0974a61d5130","Type":"ContainerStarted","Data":"7d0ed41123ccf977ad0bf80b9e150946973d44b455fab8b81f29c9f91c7cd527"} Jul 18 01:53:43.343964 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:43.343928 2580 generic.go:356] "Generic (PLEG): container finished" podID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerID="a04168c19823eee1b2077ed8ca11dc2555bdd7405aaa3202ea9c3eff8808ee55" exitCode=0 Jul 18 01:53:43.344341 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:43.344016 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" event={"ID":"ee07fcbc-cc03-45ac-8237-0974a61d5130","Type":"ContainerDied","Data":"a04168c19823eee1b2077ed8ca11dc2555bdd7405aaa3202ea9c3eff8808ee55"} Jul 18 01:53:44.350052 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:44.350016 2580 generic.go:356] "Generic (PLEG): container finished" podID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerID="df4fb68b2dd848b729e4599b9f165dd231a8bb1c367964151a50266524ef9d79" exitCode=0 Jul 18 01:53:44.350424 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:44.350077 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" event={"ID":"ee07fcbc-cc03-45ac-8237-0974a61d5130","Type":"ContainerDied","Data":"df4fb68b2dd848b729e4599b9f165dd231a8bb1c367964151a50266524ef9d79"} Jul 18 01:53:45.483655 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.483630 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:45.622687 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.622575 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-bundle\") pod \"ee07fcbc-cc03-45ac-8237-0974a61d5130\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " Jul 18 01:53:45.622687 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.622658 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76q8c\" (UniqueName: \"kubernetes.io/projected/ee07fcbc-cc03-45ac-8237-0974a61d5130-kube-api-access-76q8c\") pod \"ee07fcbc-cc03-45ac-8237-0974a61d5130\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " Jul 18 01:53:45.622909 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.622741 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-util\") pod \"ee07fcbc-cc03-45ac-8237-0974a61d5130\" (UID: \"ee07fcbc-cc03-45ac-8237-0974a61d5130\") " Jul 18 01:53:45.623321 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.623279 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-bundle" (OuterVolumeSpecName: "bundle") pod "ee07fcbc-cc03-45ac-8237-0974a61d5130" (UID: "ee07fcbc-cc03-45ac-8237-0974a61d5130"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:45.624794 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.624766 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee07fcbc-cc03-45ac-8237-0974a61d5130-kube-api-access-76q8c" (OuterVolumeSpecName: "kube-api-access-76q8c") pod "ee07fcbc-cc03-45ac-8237-0974a61d5130" (UID: "ee07fcbc-cc03-45ac-8237-0974a61d5130"). InnerVolumeSpecName "kube-api-access-76q8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:53:45.628293 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.628261 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-util" (OuterVolumeSpecName: "util") pod "ee07fcbc-cc03-45ac-8237-0974a61d5130" (UID: "ee07fcbc-cc03-45ac-8237-0974a61d5130"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:45.723292 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.723260 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:45.723292 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.723288 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee07fcbc-cc03-45ac-8237-0974a61d5130-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:45.723292 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:45.723298 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76q8c\" (UniqueName: \"kubernetes.io/projected/ee07fcbc-cc03-45ac-8237-0974a61d5130-kube-api-access-76q8c\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:53:46.364427 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:46.364398 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" Jul 18 01:53:46.364427 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:46.364406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afnjbc8" event={"ID":"ee07fcbc-cc03-45ac-8237-0974a61d5130","Type":"ContainerDied","Data":"7d0ed41123ccf977ad0bf80b9e150946973d44b455fab8b81f29c9f91c7cd527"} Jul 18 01:53:46.364648 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:46.364446 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d0ed41123ccf977ad0bf80b9e150946973d44b455fab8b81f29c9f91c7cd527" Jul 18 01:53:58.637955 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.637915 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h"] Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638183 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="util" Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638194 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="util" Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638205 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="pull" Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638211 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="pull" Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638225 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="extract" Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638231 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="extract" Jul 18 01:53:58.638398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.638278 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee07fcbc-cc03-45ac-8237-0974a61d5130" containerName="extract" Jul 18 01:53:58.645118 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.645099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.652000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.651863 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 18 01:53:58.652137 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.652120 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dfvpq\"" Jul 18 01:53:58.652212 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.652128 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 18 01:53:58.652212 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.652146 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 18 01:53:58.658213 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.658184 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h"] Jul 18 01:53:58.823434 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.823394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4360e211-4224-4eed-a262-50aba312e148-manager-config\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.823434 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.823432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqvpr\" (UniqueName: \"kubernetes.io/projected/4360e211-4224-4eed-a262-50aba312e148-kube-api-access-dqvpr\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.823698 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.823517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4360e211-4224-4eed-a262-50aba312e148-cert\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.823698 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.823606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4360e211-4224-4eed-a262-50aba312e148-metrics-cert\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.924328 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.924229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4360e211-4224-4eed-a262-50aba312e148-manager-config\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.924328 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.924268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqvpr\" (UniqueName: \"kubernetes.io/projected/4360e211-4224-4eed-a262-50aba312e148-kube-api-access-dqvpr\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.924328 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.924301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4360e211-4224-4eed-a262-50aba312e148-cert\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.924328 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.924321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4360e211-4224-4eed-a262-50aba312e148-metrics-cert\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.925066 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.925043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4360e211-4224-4eed-a262-50aba312e148-manager-config\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.926933 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.926902 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4360e211-4224-4eed-a262-50aba312e148-metrics-cert\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.926933 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.926927 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4360e211-4224-4eed-a262-50aba312e148-cert\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.936227 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.936205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqvpr\" (UniqueName: \"kubernetes.io/projected/4360e211-4224-4eed-a262-50aba312e148-kube-api-access-dqvpr\") pod \"lws-controller-manager-6bc78dbd6c-mx97h\" (UID: \"4360e211-4224-4eed-a262-50aba312e148\") " pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:58.954888 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:58.954855 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:53:59.101860 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.101834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h"] Jul 18 01:53:59.103416 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:53:59.103389 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4360e211_4224_4eed_a262_50aba312e148.slice/crio-b68afce4b030f08130677863810173204126431100e0acdf7c9d4e2d203f3ee5 WatchSource:0}: Error finding container b68afce4b030f08130677863810173204126431100e0acdf7c9d4e2d203f3ee5: Status 404 returned error can't find the container with id b68afce4b030f08130677863810173204126431100e0acdf7c9d4e2d203f3ee5 Jul 18 01:53:59.312289 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.312255 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4"] Jul 18 01:53:59.317621 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.317597 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.320861 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.320841 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 01:53:59.321149 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.321132 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 01:53:59.321383 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.321367 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hw5p9\"" Jul 18 01:53:59.321485 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.321399 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 01:53:59.321683 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.321668 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 01:53:59.419032 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.419002 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" event={"ID":"4360e211-4224-4eed-a262-50aba312e148","Type":"ContainerStarted","Data":"b68afce4b030f08130677863810173204126431100e0acdf7c9d4e2d203f3ee5"} Jul 18 01:53:59.429325 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.429301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e662e545-d50a-4f26-9326-04b3bcb0d6ae-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.429402 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.429345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e662e545-d50a-4f26-9326-04b3bcb0d6ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.429402 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.429364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgcj6\" (UniqueName: \"kubernetes.io/projected/e662e545-d50a-4f26-9326-04b3bcb0d6ae-kube-api-access-vgcj6\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.429474 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.429401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e662e545-d50a-4f26-9326-04b3bcb0d6ae-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.429474 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.429421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e662e545-d50a-4f26-9326-04b3bcb0d6ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.441346 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.441322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4"] Jul 18 01:53:59.530736 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.530706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e662e545-d50a-4f26-9326-04b3bcb0d6ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.530912 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.530757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e662e545-d50a-4f26-9326-04b3bcb0d6ae-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.530912 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.530882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e662e545-d50a-4f26-9326-04b3bcb0d6ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.530999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.530921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgcj6\" (UniqueName: \"kubernetes.io/projected/e662e545-d50a-4f26-9326-04b3bcb0d6ae-kube-api-access-vgcj6\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.530999 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.530948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e662e545-d50a-4f26-9326-04b3bcb0d6ae-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.531304 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.531274 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e662e545-d50a-4f26-9326-04b3bcb0d6ae-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.533165 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.533142 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e662e545-d50a-4f26-9326-04b3bcb0d6ae-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.533240 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.533221 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e662e545-d50a-4f26-9326-04b3bcb0d6ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.533333 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.533318 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e662e545-d50a-4f26-9326-04b3bcb0d6ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.553466 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.553447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgcj6\" (UniqueName: \"kubernetes.io/projected/e662e545-d50a-4f26-9326-04b3bcb0d6ae-kube-api-access-vgcj6\") pod \"opendatahub-operator-controller-manager-84c588c4c7-dxdl4\" (UID: \"e662e545-d50a-4f26-9326-04b3bcb0d6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.627651 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.627557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:53:59.805139 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:53:59.805112 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4"] Jul 18 01:53:59.808011 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:53:59.807974 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode662e545_d50a_4f26_9326_04b3bcb0d6ae.slice/crio-06beda4812b6be759167c49dd76b5e71d7124376d596d3482f44d447ea5732b7 WatchSource:0}: Error finding container 06beda4812b6be759167c49dd76b5e71d7124376d596d3482f44d447ea5732b7: Status 404 returned error can't find the container with id 06beda4812b6be759167c49dd76b5e71d7124376d596d3482f44d447ea5732b7 Jul 18 01:54:00.427868 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:00.427748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" event={"ID":"e662e545-d50a-4f26-9326-04b3bcb0d6ae","Type":"ContainerStarted","Data":"06beda4812b6be759167c49dd76b5e71d7124376d596d3482f44d447ea5732b7"} Jul 18 01:54:01.435049 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:01.435012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" event={"ID":"4360e211-4224-4eed-a262-50aba312e148","Type":"ContainerStarted","Data":"27aaa055baafcf2f5281e6a87371f99d07aedff740c0cfd874e5d6da4508294b"} Jul 18 01:54:01.435464 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:01.435120 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:54:01.469132 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:01.469070 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" podStartSLOduration=1.884910334 podStartE2EDuration="3.469051994s" totalImagesPullingTime="1.58414166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:01.467110931 +0000 UTC m=+347.150983670" watchObservedRunningTime="2026-07-18 01:54:01.469051994 +0000 UTC m=+347.152924731" Jul 18 01:54:03.446319 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:03.446282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" event={"ID":"e662e545-d50a-4f26-9326-04b3bcb0d6ae","Type":"ContainerStarted","Data":"0b47f252835e33359aaaab433c36633ef2b8526dee32099b5a25a672328df1cd"} Jul 18 01:54:04.451667 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:04.451629 2580 generic.go:356] "Generic (PLEG): container finished" podID="e662e545-d50a-4f26-9326-04b3bcb0d6ae" containerID="0b47f252835e33359aaaab433c36633ef2b8526dee32099b5a25a672328df1cd" exitCode=0 Jul 18 01:54:04.452149 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:04.451680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" event={"ID":"e662e545-d50a-4f26-9326-04b3bcb0d6ae","Type":"ContainerDied","Data":"0b47f252835e33359aaaab433c36633ef2b8526dee32099b5a25a672328df1cd"} Jul 18 01:54:05.457993 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:05.457955 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" event={"ID":"e662e545-d50a-4f26-9326-04b3bcb0d6ae","Type":"ContainerStarted","Data":"a76a142a9486f08c1d06e225bc6923541d46afa08bba397ec6691ceeca7d0da5"} Jul 18 01:54:05.458362 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:05.458084 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:54:05.513994 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:05.513939 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" podStartSLOduration=2.079706886 podStartE2EDuration="6.513923067s" totalImagesPullingTime="3.431548611s" totalInitContainerRuntime="1.00266757s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:05.508099496 +0000 UTC m=+351.191972235" watchObservedRunningTime="2026-07-18 01:54:05.513923067 +0000 UTC m=+351.197795805" Jul 18 01:54:12.442384 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:12.442310 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6bc78dbd6c-mx97h" Jul 18 01:54:16.464552 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:16.464519 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-dxdl4" Jul 18 01:54:26.952500 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.952462 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt"] Jul 18 01:54:26.960957 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.960928 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:26.963827 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.963809 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-vg92x\"" Jul 18 01:54:26.964165 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.964151 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 01:54:26.964567 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.964552 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 01:54:26.964770 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.964755 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 01:54:26.993014 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:26.992990 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt"] Jul 18 01:54:27.022416 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022513 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022513 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022513 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2kqz\" (UniqueName: \"kubernetes.io/projected/70bce6bf-7ca1-467f-a43e-f9872de70801-kube-api-access-t2kqz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022668 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022525 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022668 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022668 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022668 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.022668 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.022657 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70bce6bf-7ca1-467f-a43e-f9872de70801-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123400 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2kqz\" (UniqueName: \"kubernetes.io/projected/70bce6bf-7ca1-467f-a43e-f9872de70801-kube-api-access-t2kqz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.123624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70bce6bf-7ca1-467f-a43e-f9872de70801-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.124062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.124062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.123909 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.124062 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.124002 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.124200 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.124116 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.124299 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.124280 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70bce6bf-7ca1-467f-a43e-f9872de70801-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.125981 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.125951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.126076 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.126029 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.135928 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.135903 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2kqz\" (UniqueName: \"kubernetes.io/projected/70bce6bf-7ca1-467f-a43e-f9872de70801-kube-api-access-t2kqz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.136350 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.136330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70bce6bf-7ca1-467f-a43e-f9872de70801-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bnzwt\" (UID: \"70bce6bf-7ca1-467f-a43e-f9872de70801\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.271171 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.271131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:27.422141 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.422116 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt"] Jul 18 01:54:27.424090 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:54:27.424064 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70bce6bf_7ca1_467f_a43e_f9872de70801.slice/crio-d92731d94562ba8e0b40d45df661883f9a50f879ab3103475ebc356c48756977 WatchSource:0}: Error finding container d92731d94562ba8e0b40d45df661883f9a50f879ab3103475ebc356c48756977: Status 404 returned error can't find the container with id d92731d94562ba8e0b40d45df661883f9a50f879ab3103475ebc356c48756977 Jul 18 01:54:27.553065 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.553002 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" event={"ID":"70bce6bf-7ca1-467f-a43e-f9872de70801","Type":"ContainerStarted","Data":"d92731d94562ba8e0b40d45df661883f9a50f879ab3103475ebc356c48756977"} Jul 18 01:54:27.906060 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.905983 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8"] Jul 18 01:54:27.909201 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.909180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:27.916080 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.916061 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 18 01:54:27.916268 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.916069 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r9btw\"" Jul 18 01:54:27.916715 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.916692 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 18 01:54:27.916835 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.916737 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 18 01:54:27.929996 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:27.929973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8"] Jul 18 01:54:28.031327 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/020daee4-bbfd-4358-913d-53a36cd4935c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.031837 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031352 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.031837 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8524h\" (UniqueName: \"kubernetes.io/projected/020daee4-bbfd-4358-913d-53a36cd4935c-kube-api-access-8524h\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.031837 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.031837 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/020daee4-bbfd-4358-913d-53a36cd4935c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.031837 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.031837 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.031605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/020daee4-bbfd-4358-913d-53a36cd4935c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.132998 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.132965 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8524h\" (UniqueName: \"kubernetes.io/projected/020daee4-bbfd-4358-913d-53a36cd4935c-kube-api-access-8524h\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.133162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.133015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.133162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.133043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/020daee4-bbfd-4358-913d-53a36cd4935c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.133162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.133084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.133162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.133109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/020daee4-bbfd-4358-913d-53a36cd4935c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.133162 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.133157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/020daee4-bbfd-4358-913d-53a36cd4935c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.133431 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.133189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.136232 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.136199 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.136955 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.136904 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/020daee4-bbfd-4358-913d-53a36cd4935c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.137198 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.137176 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.137274 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.137181 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/020daee4-bbfd-4358-913d-53a36cd4935c-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.139118 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.139096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/020daee4-bbfd-4358-913d-53a36cd4935c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.144065 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.144013 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/020daee4-bbfd-4358-913d-53a36cd4935c-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.144398 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.144378 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8524h\" (UniqueName: \"kubernetes.io/projected/020daee4-bbfd-4358-913d-53a36cd4935c-kube-api-access-8524h\") pod \"istiod-openshift-gateway-94bb8fbfd-jmmc8\" (UID: \"020daee4-bbfd-4358-913d-53a36cd4935c\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.221075 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.221007 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:28.454663 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.454631 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8"] Jul 18 01:54:28.458183 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:54:28.458153 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020daee4_bbfd_4358_913d_53a36cd4935c.slice/crio-972e4ae8827aa4e020a7c3230bbaba9ecd327f2ddf2d91cdea9975c2fe8bc8db WatchSource:0}: Error finding container 972e4ae8827aa4e020a7c3230bbaba9ecd327f2ddf2d91cdea9975c2fe8bc8db: Status 404 returned error can't find the container with id 972e4ae8827aa4e020a7c3230bbaba9ecd327f2ddf2d91cdea9975c2fe8bc8db Jul 18 01:54:28.560014 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:28.559968 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" event={"ID":"020daee4-bbfd-4358-913d-53a36cd4935c","Type":"ContainerStarted","Data":"972e4ae8827aa4e020a7c3230bbaba9ecd327f2ddf2d91cdea9975c2fe8bc8db"} Jul 18 01:54:29.913667 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:29.913630 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 01:54:29.913925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:29.913704 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 01:54:29.913925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:29.913732 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 01:54:30.572428 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:30.572384 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" event={"ID":"70bce6bf-7ca1-467f-a43e-f9872de70801","Type":"ContainerStarted","Data":"2db1fa23e1e6587b33d8677c32eaaa15c725f188287ad346110cca517c685eea"} Jul 18 01:54:30.637251 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:30.637194 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" podStartSLOduration=2.149852081 podStartE2EDuration="4.637174216s" totalImagesPullingTime="2.487322135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:30.636185808 +0000 UTC m=+376.320058547" watchObservedRunningTime="2026-07-18 01:54:30.637174216 +0000 UTC m=+376.321046957" Jul 18 01:54:31.274516 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:31.274472 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:31.281011 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:31.280983 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:31.577705 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:31.577630 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:31.578524 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:31.578506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bnzwt" Jul 18 01:54:31.664950 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:31.664911 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 01:54:31.665039 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:31.664973 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 01:54:32.583738 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:32.583706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" event={"ID":"020daee4-bbfd-4358-913d-53a36cd4935c","Type":"ContainerStarted","Data":"dfb568cc118465d36b065f8c4c4e3baa18d9c293273769cd23cf3bb13e9a3080"} Jul 18 01:54:32.619769 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:32.619720 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" podStartSLOduration=2.41553236 podStartE2EDuration="5.619703192s" totalImagesPullingTime="3.204170832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:32.618006932 +0000 UTC m=+378.301879671" watchObservedRunningTime="2026-07-18 01:54:32.619703192 +0000 UTC m=+378.303575932" Jul 18 01:54:33.588750 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:33.588716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:54:33.590636 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:54:33.590614 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-jmmc8" Jul 18 01:55:36.046529 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.046492 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4"] Jul 18 01:55:36.049373 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.049349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.051732 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.051707 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qrwf8\"" Jul 18 01:55:36.051861 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.051709 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 01:55:36.051861 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.051711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 01:55:36.057959 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.057933 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4"] Jul 18 01:55:36.165135 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.165101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.165308 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.165159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df5fj\" (UniqueName: \"kubernetes.io/projected/4db124ea-af61-4c50-a582-8a4a33ca15db-kube-api-access-df5fj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.165308 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.165188 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.265777 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.265728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.265961 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.265848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.265961 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.265906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df5fj\" (UniqueName: \"kubernetes.io/projected/4db124ea-af61-4c50-a582-8a4a33ca15db-kube-api-access-df5fj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.266205 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.266186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.266241 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.266209 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.275964 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.275930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df5fj\" (UniqueName: \"kubernetes.io/projected/4db124ea-af61-4c50-a582-8a4a33ca15db-kube-api-access-df5fj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.359810 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.359716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:36.503171 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.503145 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4"] Jul 18 01:55:36.505231 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:55:36.505207 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4db124ea_af61_4c50_a582_8a4a33ca15db.slice/crio-dae4f684b2980bc7026ac1ea4e55d25ca328fbd2f98bf34fbc2ae6c0e9513ced WatchSource:0}: Error finding container dae4f684b2980bc7026ac1ea4e55d25ca328fbd2f98bf34fbc2ae6c0e9513ced: Status 404 returned error can't find the container with id dae4f684b2980bc7026ac1ea4e55d25ca328fbd2f98bf34fbc2ae6c0e9513ced Jul 18 01:55:36.633736 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.633657 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p"] Jul 18 01:55:36.635991 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.635972 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.648351 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.648322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p"] Jul 18 01:55:36.770479 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.770437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qdnp\" (UniqueName: \"kubernetes.io/projected/08d635bd-bdde-4223-912e-c5f69936aba4-kube-api-access-4qdnp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.770712 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.770571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.770712 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.770653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.869860 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.869823 2580 generic.go:356] "Generic (PLEG): container finished" podID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerID="2ea7b9e577f1bec9092a8f255dd09558ca6acc4022936ee991d981780e52c2fa" exitCode=0 Jul 18 01:55:36.870013 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.869875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" event={"ID":"4db124ea-af61-4c50-a582-8a4a33ca15db","Type":"ContainerDied","Data":"2ea7b9e577f1bec9092a8f255dd09558ca6acc4022936ee991d981780e52c2fa"} Jul 18 01:55:36.870013 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.869920 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" event={"ID":"4db124ea-af61-4c50-a582-8a4a33ca15db","Type":"ContainerStarted","Data":"dae4f684b2980bc7026ac1ea4e55d25ca328fbd2f98bf34fbc2ae6c0e9513ced"} Jul 18 01:55:36.870994 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.870970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qdnp\" (UniqueName: \"kubernetes.io/projected/08d635bd-bdde-4223-912e-c5f69936aba4-kube-api-access-4qdnp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.871200 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.871178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.871285 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.871228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.871549 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.871533 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.871599 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.871561 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.878635 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.878577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qdnp\" (UniqueName: \"kubernetes.io/projected/08d635bd-bdde-4223-912e-c5f69936aba4-kube-api-access-4qdnp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:36.975130 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:36.975099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:37.070925 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.070889 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4"] Jul 18 01:55:37.075124 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.075100 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.083892 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.083867 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4"] Jul 18 01:55:37.116229 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.116199 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p"] Jul 18 01:55:37.117997 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:55:37.117970 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d635bd_bdde_4223_912e_c5f69936aba4.slice/crio-01997f59c702cc9f2a60437f8910402bbc2ff0bb57c353b8659c45cc88a1d3ad WatchSource:0}: Error finding container 01997f59c702cc9f2a60437f8910402bbc2ff0bb57c353b8659c45cc88a1d3ad: Status 404 returned error can't find the container with id 01997f59c702cc9f2a60437f8910402bbc2ff0bb57c353b8659c45cc88a1d3ad Jul 18 01:55:37.173140 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.173109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.173262 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.173164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.173262 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.173193 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mtt4\" (UniqueName: \"kubernetes.io/projected/b6190791-f1dc-474c-818e-977b822f4338-kube-api-access-2mtt4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.274125 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.274039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.274125 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.274086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.274125 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.274112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mtt4\" (UniqueName: \"kubernetes.io/projected/b6190791-f1dc-474c-818e-977b822f4338-kube-api-access-2mtt4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.274432 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.274407 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.274501 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.274477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.283550 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.283508 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mtt4\" (UniqueName: \"kubernetes.io/projected/b6190791-f1dc-474c-818e-977b822f4338-kube-api-access-2mtt4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.386203 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.386162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:37.428989 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.428951 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w"] Jul 18 01:55:37.433197 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.433174 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.440691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.440622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w"] Jul 18 01:55:37.534866 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.534840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4"] Jul 18 01:55:37.544835 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:55:37.544806 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6190791_f1dc_474c_818e_977b822f4338.slice/crio-7dacf791dada333666b4d2476c7002e311cd5ef242e83de7c716b602fd2c151e WatchSource:0}: Error finding container 7dacf791dada333666b4d2476c7002e311cd5ef242e83de7c716b602fd2c151e: Status 404 returned error can't find the container with id 7dacf791dada333666b4d2476c7002e311cd5ef242e83de7c716b602fd2c151e Jul 18 01:55:37.576624 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.576571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.576730 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.576654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p9mq\" (UniqueName: \"kubernetes.io/projected/49dd3fd2-70c4-4dda-8843-b4fea0352786-kube-api-access-5p9mq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.576801 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.576756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.677755 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.677710 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.677938 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.677765 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p9mq\" (UniqueName: \"kubernetes.io/projected/49dd3fd2-70c4-4dda-8843-b4fea0352786-kube-api-access-5p9mq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.677938 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.677810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.678183 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.678160 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.678256 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.678185 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.686977 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.686947 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p9mq\" (UniqueName: \"kubernetes.io/projected/49dd3fd2-70c4-4dda-8843-b4fea0352786-kube-api-access-5p9mq\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.748394 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.748349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:37.877143 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.877110 2580 generic.go:356] "Generic (PLEG): container finished" podID="08d635bd-bdde-4223-912e-c5f69936aba4" containerID="8c6d7e85f98b2dd2e59202db160ec24ab4e08e952745fa86afd598572626bd7b" exitCode=0 Jul 18 01:55:37.877420 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.877148 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" event={"ID":"08d635bd-bdde-4223-912e-c5f69936aba4","Type":"ContainerDied","Data":"8c6d7e85f98b2dd2e59202db160ec24ab4e08e952745fa86afd598572626bd7b"} Jul 18 01:55:37.877420 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.877199 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" event={"ID":"08d635bd-bdde-4223-912e-c5f69936aba4","Type":"ContainerStarted","Data":"01997f59c702cc9f2a60437f8910402bbc2ff0bb57c353b8659c45cc88a1d3ad"} Jul 18 01:55:37.878836 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.878802 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6190791-f1dc-474c-818e-977b822f4338" containerID="f32a5894878c992b02ec29c2fbcddafde54e3ee127c0cb4ba8d2970481a62376" exitCode=0 Jul 18 01:55:37.878957 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.878873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" event={"ID":"b6190791-f1dc-474c-818e-977b822f4338","Type":"ContainerDied","Data":"f32a5894878c992b02ec29c2fbcddafde54e3ee127c0cb4ba8d2970481a62376"} Jul 18 01:55:37.878957 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.878928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" event={"ID":"b6190791-f1dc-474c-818e-977b822f4338","Type":"ContainerStarted","Data":"7dacf791dada333666b4d2476c7002e311cd5ef242e83de7c716b602fd2c151e"} Jul 18 01:55:37.880856 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.880824 2580 generic.go:356] "Generic (PLEG): container finished" podID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerID="4c34a056df81adc88c1bd942a5057b9c5c426683c6136f57d5fa1dc14aeb5171" exitCode=0 Jul 18 01:55:37.880914 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.880861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" event={"ID":"4db124ea-af61-4c50-a582-8a4a33ca15db","Type":"ContainerDied","Data":"4c34a056df81adc88c1bd942a5057b9c5c426683c6136f57d5fa1dc14aeb5171"} Jul 18 01:55:37.885553 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:37.885532 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w"] Jul 18 01:55:37.887448 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:55:37.887424 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49dd3fd2_70c4_4dda_8843_b4fea0352786.slice/crio-9b5adc464b38c30f7a9db4adb53371eacf978de449e7967cb04a0fc7297a589e WatchSource:0}: Error finding container 9b5adc464b38c30f7a9db4adb53371eacf978de449e7967cb04a0fc7297a589e: Status 404 returned error can't find the container with id 9b5adc464b38c30f7a9db4adb53371eacf978de449e7967cb04a0fc7297a589e Jul 18 01:55:38.888449 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.888420 2580 generic.go:356] "Generic (PLEG): container finished" podID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerID="5bf8985a83437e43793f1ca9bae62355b573ca845884f304a402448b14433fb0" exitCode=0 Jul 18 01:55:38.888838 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.888490 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" event={"ID":"4db124ea-af61-4c50-a582-8a4a33ca15db","Type":"ContainerDied","Data":"5bf8985a83437e43793f1ca9bae62355b573ca845884f304a402448b14433fb0"} Jul 18 01:55:38.890222 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.890202 2580 generic.go:356] "Generic (PLEG): container finished" podID="08d635bd-bdde-4223-912e-c5f69936aba4" containerID="06bf4d8f4dacd7b13ee8b546f092fadd22f7c4bbb59a774f94d5c5e35ac532f1" exitCode=0 Jul 18 01:55:38.890310 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.890271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" event={"ID":"08d635bd-bdde-4223-912e-c5f69936aba4","Type":"ContainerDied","Data":"06bf4d8f4dacd7b13ee8b546f092fadd22f7c4bbb59a774f94d5c5e35ac532f1"} Jul 18 01:55:38.892000 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.891979 2580 generic.go:356] "Generic (PLEG): container finished" podID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerID="46bd33f2bca78ae457e782b0452d3b104b94d111d6bed7f122988389bc09f706" exitCode=0 Jul 18 01:55:38.894368 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.894336 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6190791-f1dc-474c-818e-977b822f4338" containerID="36d2121467de0fb831bd9639ef72ef672a509647e943fe7189f667ffd27726fb" exitCode=0 Jul 18 01:55:38.898290 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.898265 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" event={"ID":"49dd3fd2-70c4-4dda-8843-b4fea0352786","Type":"ContainerDied","Data":"46bd33f2bca78ae457e782b0452d3b104b94d111d6bed7f122988389bc09f706"} Jul 18 01:55:38.898383 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.898308 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" event={"ID":"49dd3fd2-70c4-4dda-8843-b4fea0352786","Type":"ContainerStarted","Data":"9b5adc464b38c30f7a9db4adb53371eacf978de449e7967cb04a0fc7297a589e"} Jul 18 01:55:38.898383 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:38.898331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" event={"ID":"b6190791-f1dc-474c-818e-977b822f4338","Type":"ContainerDied","Data":"36d2121467de0fb831bd9639ef72ef672a509647e943fe7189f667ffd27726fb"} Jul 18 01:55:39.901835 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:39.901802 2580 generic.go:356] "Generic (PLEG): container finished" podID="08d635bd-bdde-4223-912e-c5f69936aba4" containerID="e9250864ac1f417f1420c930582b2ca48f5b953fdc2bd019d0b538077ccd566d" exitCode=0 Jul 18 01:55:39.902255 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:39.901847 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" event={"ID":"08d635bd-bdde-4223-912e-c5f69936aba4","Type":"ContainerDied","Data":"e9250864ac1f417f1420c930582b2ca48f5b953fdc2bd019d0b538077ccd566d"} Jul 18 01:55:39.903522 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:39.903501 2580 generic.go:356] "Generic (PLEG): container finished" podID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerID="410fe4b97bdb415b178f87f7b7c6bb87f08e961cba688b32468c6655495aa312" exitCode=0 Jul 18 01:55:39.903656 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:39.903576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" event={"ID":"49dd3fd2-70c4-4dda-8843-b4fea0352786","Type":"ContainerDied","Data":"410fe4b97bdb415b178f87f7b7c6bb87f08e961cba688b32468c6655495aa312"} Jul 18 01:55:39.905947 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:39.905926 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6190791-f1dc-474c-818e-977b822f4338" containerID="31b1c632dbfee3eb78c193a0d892e251ec58714d117d0197e1f98437436fad75" exitCode=0 Jul 18 01:55:39.906028 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:39.906005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" event={"ID":"b6190791-f1dc-474c-818e-977b822f4338","Type":"ContainerDied","Data":"31b1c632dbfee3eb78c193a0d892e251ec58714d117d0197e1f98437436fad75"} Jul 18 01:55:40.035860 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.035835 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:40.199063 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.198961 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-bundle\") pod \"4db124ea-af61-4c50-a582-8a4a33ca15db\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " Jul 18 01:55:40.199063 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.199058 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-df5fj\" (UniqueName: \"kubernetes.io/projected/4db124ea-af61-4c50-a582-8a4a33ca15db-kube-api-access-df5fj\") pod \"4db124ea-af61-4c50-a582-8a4a33ca15db\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " Jul 18 01:55:40.199266 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.199106 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-util\") pod \"4db124ea-af61-4c50-a582-8a4a33ca15db\" (UID: \"4db124ea-af61-4c50-a582-8a4a33ca15db\") " Jul 18 01:55:40.199609 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.199561 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-bundle" (OuterVolumeSpecName: "bundle") pod "4db124ea-af61-4c50-a582-8a4a33ca15db" (UID: "4db124ea-af61-4c50-a582-8a4a33ca15db"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:40.201260 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.201235 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4db124ea-af61-4c50-a582-8a4a33ca15db-kube-api-access-df5fj" (OuterVolumeSpecName: "kube-api-access-df5fj") pod "4db124ea-af61-4c50-a582-8a4a33ca15db" (UID: "4db124ea-af61-4c50-a582-8a4a33ca15db"). InnerVolumeSpecName "kube-api-access-df5fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:40.204737 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.204716 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-util" (OuterVolumeSpecName: "util") pod "4db124ea-af61-4c50-a582-8a4a33ca15db" (UID: "4db124ea-af61-4c50-a582-8a4a33ca15db"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:40.300424 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.300389 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df5fj\" (UniqueName: \"kubernetes.io/projected/4db124ea-af61-4c50-a582-8a4a33ca15db-kube-api-access-df5fj\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:40.300424 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.300420 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:40.300573 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.300437 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4db124ea-af61-4c50-a582-8a4a33ca15db-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:40.912378 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.912352 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" Jul 18 01:55:40.912785 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.912356 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4" event={"ID":"4db124ea-af61-4c50-a582-8a4a33ca15db","Type":"ContainerDied","Data":"dae4f684b2980bc7026ac1ea4e55d25ca328fbd2f98bf34fbc2ae6c0e9513ced"} Jul 18 01:55:40.912785 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.912477 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dae4f684b2980bc7026ac1ea4e55d25ca328fbd2f98bf34fbc2ae6c0e9513ced" Jul 18 01:55:40.914711 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.914684 2580 generic.go:356] "Generic (PLEG): container finished" podID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerID="473cd32ea89745bfd9a0dc211873930d5b38be3d2db1f5740b4d15f07f8f99f7" exitCode=0 Jul 18 01:55:40.914832 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:40.914780 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" event={"ID":"49dd3fd2-70c4-4dda-8843-b4fea0352786","Type":"ContainerDied","Data":"473cd32ea89745bfd9a0dc211873930d5b38be3d2db1f5740b4d15f07f8f99f7"} Jul 18 01:55:41.046560 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.046536 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:41.080273 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.080244 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:41.209572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.209478 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-bundle\") pod \"08d635bd-bdde-4223-912e-c5f69936aba4\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " Jul 18 01:55:41.209572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.209516 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-util\") pod \"08d635bd-bdde-4223-912e-c5f69936aba4\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " Jul 18 01:55:41.209572 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.209569 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-bundle\") pod \"b6190791-f1dc-474c-818e-977b822f4338\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " Jul 18 01:55:41.209856 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.209639 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-util\") pod \"b6190791-f1dc-474c-818e-977b822f4338\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " Jul 18 01:55:41.209856 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.209681 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mtt4\" (UniqueName: \"kubernetes.io/projected/b6190791-f1dc-474c-818e-977b822f4338-kube-api-access-2mtt4\") pod \"b6190791-f1dc-474c-818e-977b822f4338\" (UID: \"b6190791-f1dc-474c-818e-977b822f4338\") " Jul 18 01:55:41.209856 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.209715 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qdnp\" (UniqueName: \"kubernetes.io/projected/08d635bd-bdde-4223-912e-c5f69936aba4-kube-api-access-4qdnp\") pod \"08d635bd-bdde-4223-912e-c5f69936aba4\" (UID: \"08d635bd-bdde-4223-912e-c5f69936aba4\") " Jul 18 01:55:41.210159 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.210123 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-bundle" (OuterVolumeSpecName: "bundle") pod "b6190791-f1dc-474c-818e-977b822f4338" (UID: "b6190791-f1dc-474c-818e-977b822f4338"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:41.210248 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.210167 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-bundle" (OuterVolumeSpecName: "bundle") pod "08d635bd-bdde-4223-912e-c5f69936aba4" (UID: "08d635bd-bdde-4223-912e-c5f69936aba4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:41.211880 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.211858 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d635bd-bdde-4223-912e-c5f69936aba4-kube-api-access-4qdnp" (OuterVolumeSpecName: "kube-api-access-4qdnp") pod "08d635bd-bdde-4223-912e-c5f69936aba4" (UID: "08d635bd-bdde-4223-912e-c5f69936aba4"). InnerVolumeSpecName "kube-api-access-4qdnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:41.211950 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.211899 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6190791-f1dc-474c-818e-977b822f4338-kube-api-access-2mtt4" (OuterVolumeSpecName: "kube-api-access-2mtt4") pod "b6190791-f1dc-474c-818e-977b822f4338" (UID: "b6190791-f1dc-474c-818e-977b822f4338"). InnerVolumeSpecName "kube-api-access-2mtt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:41.218370 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.218341 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-util" (OuterVolumeSpecName: "util") pod "b6190791-f1dc-474c-818e-977b822f4338" (UID: "b6190791-f1dc-474c-818e-977b822f4338"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:41.218774 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.218757 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-util" (OuterVolumeSpecName: "util") pod "08d635bd-bdde-4223-912e-c5f69936aba4" (UID: "08d635bd-bdde-4223-912e-c5f69936aba4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:41.311157 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.311123 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:41.311157 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.311151 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b6190791-f1dc-474c-818e-977b822f4338-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:41.311157 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.311161 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mtt4\" (UniqueName: \"kubernetes.io/projected/b6190791-f1dc-474c-818e-977b822f4338-kube-api-access-2mtt4\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:41.311356 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.311171 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qdnp\" (UniqueName: \"kubernetes.io/projected/08d635bd-bdde-4223-912e-c5f69936aba4-kube-api-access-4qdnp\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:41.311356 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.311181 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:41.311356 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.311188 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08d635bd-bdde-4223-912e-c5f69936aba4-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:41.921660 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.921610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" event={"ID":"08d635bd-bdde-4223-912e-c5f69936aba4","Type":"ContainerDied","Data":"01997f59c702cc9f2a60437f8910402bbc2ff0bb57c353b8659c45cc88a1d3ad"} Jul 18 01:55:41.921660 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.921661 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p" Jul 18 01:55:41.922165 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.921675 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01997f59c702cc9f2a60437f8910402bbc2ff0bb57c353b8659c45cc88a1d3ad" Jul 18 01:55:41.923505 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.923486 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" Jul 18 01:55:41.923610 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.923498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4" event={"ID":"b6190791-f1dc-474c-818e-977b822f4338","Type":"ContainerDied","Data":"7dacf791dada333666b4d2476c7002e311cd5ef242e83de7c716b602fd2c151e"} Jul 18 01:55:41.923610 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:41.923538 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dacf791dada333666b4d2476c7002e311cd5ef242e83de7c716b602fd2c151e" Jul 18 01:55:42.052130 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.052098 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:42.218653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.218560 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-util\") pod \"49dd3fd2-70c4-4dda-8843-b4fea0352786\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " Jul 18 01:55:42.218810 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.218682 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p9mq\" (UniqueName: \"kubernetes.io/projected/49dd3fd2-70c4-4dda-8843-b4fea0352786-kube-api-access-5p9mq\") pod \"49dd3fd2-70c4-4dda-8843-b4fea0352786\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " Jul 18 01:55:42.218810 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.218703 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-bundle\") pod \"49dd3fd2-70c4-4dda-8843-b4fea0352786\" (UID: \"49dd3fd2-70c4-4dda-8843-b4fea0352786\") " Jul 18 01:55:42.219271 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.219236 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-bundle" (OuterVolumeSpecName: "bundle") pod "49dd3fd2-70c4-4dda-8843-b4fea0352786" (UID: "49dd3fd2-70c4-4dda-8843-b4fea0352786"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:42.220869 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.220847 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49dd3fd2-70c4-4dda-8843-b4fea0352786-kube-api-access-5p9mq" (OuterVolumeSpecName: "kube-api-access-5p9mq") pod "49dd3fd2-70c4-4dda-8843-b4fea0352786" (UID: "49dd3fd2-70c4-4dda-8843-b4fea0352786"). InnerVolumeSpecName "kube-api-access-5p9mq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:42.223859 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.223834 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-util" (OuterVolumeSpecName: "util") pod "49dd3fd2-70c4-4dda-8843-b4fea0352786" (UID: "49dd3fd2-70c4-4dda-8843-b4fea0352786"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:42.320196 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.320157 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-util\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:42.320196 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.320190 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p9mq\" (UniqueName: \"kubernetes.io/projected/49dd3fd2-70c4-4dda-8843-b4fea0352786-kube-api-access-5p9mq\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:42.320196 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.320201 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49dd3fd2-70c4-4dda-8843-b4fea0352786-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:55:42.929775 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.929747 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" Jul 18 01:55:42.930159 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.929746 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w" event={"ID":"49dd3fd2-70c4-4dda-8843-b4fea0352786","Type":"ContainerDied","Data":"9b5adc464b38c30f7a9db4adb53371eacf978de449e7967cb04a0fc7297a589e"} Jul 18 01:55:42.930159 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:55:42.929871 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b5adc464b38c30f7a9db4adb53371eacf978de449e7967cb04a0fc7297a589e" Jul 18 01:56:05.625738 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.625708 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f8db66bcb-vvht9"] Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.625992 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="util" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626002 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="util" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626011 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626018 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626030 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="util" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626035 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="util" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626042 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="pull" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626048 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="pull" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626053 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="pull" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626058 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="pull" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626066 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="pull" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626070 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="pull" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626077 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626098 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626106 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626113 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626120 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="extract" Jul 18 01:56:05.626119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626125 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="extract" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626132 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="pull" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626137 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="pull" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626145 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="util" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626150 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="util" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626156 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="util" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626161 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="util" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626208 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4db124ea-af61-4c50-a582-8a4a33ca15db" containerName="extract" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626214 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d635bd-bdde-4223-912e-c5f69936aba4" containerName="extract" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626219 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6190791-f1dc-474c-818e-977b822f4338" containerName="extract" Jul 18 01:56:05.626691 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.626226 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49dd3fd2-70c4-4dda-8843-b4fea0352786" containerName="extract" Jul 18 01:56:05.630379 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.630351 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.642309 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.642287 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8db66bcb-vvht9"] Jul 18 01:56:05.707551 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707509 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-serving-cert\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.707551 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-config\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.707777 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-trusted-ca-bundle\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.707777 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707674 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-oauth-config\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.707777 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-oauth-serving-cert\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.707777 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707746 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-service-ca\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.707913 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.707779 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtpjm\" (UniqueName: \"kubernetes.io/projected/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-kube-api-access-qtpjm\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809098 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtpjm\" (UniqueName: \"kubernetes.io/projected/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-kube-api-access-qtpjm\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809303 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809118 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-serving-cert\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809303 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-config\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809303 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-trusted-ca-bundle\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809303 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-oauth-config\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809500 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-oauth-serving-cert\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.809545 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-service-ca\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.810002 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.809976 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-config\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.810202 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.810177 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-trusted-ca-bundle\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.810287 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.810223 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-oauth-serving-cert\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.810287 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.810238 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-service-ca\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.811574 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.811544 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-serving-cert\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.811748 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.811729 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-console-oauth-config\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.826117 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.826094 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtpjm\" (UniqueName: \"kubernetes.io/projected/fbf89c88-bc0c-457b-aa05-366ed0e8c39d-kube-api-access-qtpjm\") pod \"console-7f8db66bcb-vvht9\" (UID: \"fbf89c88-bc0c-457b-aa05-366ed0e8c39d\") " pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:05.939077 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:05.938988 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:06.112695 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:06.112663 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f8db66bcb-vvht9"] Jul 18 01:56:06.113750 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:56:06.113725 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf89c88_bc0c_457b_aa05_366ed0e8c39d.slice/crio-fdc9e7cad9f58692f4390dca84c7e5350fc721edee70b091caa828047c4c88d2 WatchSource:0}: Error finding container fdc9e7cad9f58692f4390dca84c7e5350fc721edee70b091caa828047c4c88d2: Status 404 returned error can't find the container with id fdc9e7cad9f58692f4390dca84c7e5350fc721edee70b091caa828047c4c88d2 Jul 18 01:56:07.042751 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:07.042715 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8db66bcb-vvht9" event={"ID":"fbf89c88-bc0c-457b-aa05-366ed0e8c39d","Type":"ContainerStarted","Data":"5e699c2359a5a88222dcdd20e0eb7ceecf7dfda6a30306e6bd6189cd6a02523b"} Jul 18 01:56:07.043227 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:07.042764 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f8db66bcb-vvht9" event={"ID":"fbf89c88-bc0c-457b-aa05-366ed0e8c39d","Type":"ContainerStarted","Data":"fdc9e7cad9f58692f4390dca84c7e5350fc721edee70b091caa828047c4c88d2"} Jul 18 01:56:07.067945 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:07.067892 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f8db66bcb-vvht9" podStartSLOduration=2.067875375 podStartE2EDuration="2.067875375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:07.066530546 +0000 UTC m=+472.750403284" watchObservedRunningTime="2026-07-18 01:56:07.067875375 +0000 UTC m=+472.751748150" Jul 18 01:56:15.939625 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:15.939574 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:15.940003 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:15.939683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:15.944419 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:15.944397 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:16.089073 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:16.089042 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f8db66bcb-vvht9" Jul 18 01:56:16.149372 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:16.149332 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f887f797d-7lhxd"] Jul 18 01:56:41.170342 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.170303 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f887f797d-7lhxd" podUID="b8cd3c46-9a6d-4371-adc3-d0e65dee6826" containerName="console" containerID="cri-o://1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0" gracePeriod=15 Jul 18 01:56:41.416276 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.416251 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:56:41.515262 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515228 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-config\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515397 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515294 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-serving-cert\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515397 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515339 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-service-ca\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515397 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515371 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-trusted-ca-bundle\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515408 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-oauth-config\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515443 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-oauth-serving-cert\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515557 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515480 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc87s\" (UniqueName: \"kubernetes.io/projected/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-kube-api-access-cc87s\") pod \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\" (UID: \"b8cd3c46-9a6d-4371-adc3-d0e65dee6826\") " Jul 18 01:56:41.515733 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515660 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-config" (OuterVolumeSpecName: "console-config") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:56:41.515836 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515809 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:56:41.516114 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515861 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:41.516114 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.515900 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:56:41.516253 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.516168 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:56:41.517527 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.517499 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:56:41.517971 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.517938 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:56:41.517971 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.517949 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-kube-api-access-cc87s" (OuterVolumeSpecName: "kube-api-access-cc87s") pod "b8cd3c46-9a6d-4371-adc3-d0e65dee6826" (UID: "b8cd3c46-9a6d-4371-adc3-d0e65dee6826"). InnerVolumeSpecName "kube-api-access-cc87s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:41.616521 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.616489 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:41.616521 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.616516 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-service-ca\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:41.616521 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.616527 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-trusted-ca-bundle\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:41.616766 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.616535 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-console-oauth-config\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:41.616766 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.616543 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-oauth-serving-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:41.616766 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:41.616552 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc87s\" (UniqueName: \"kubernetes.io/projected/b8cd3c46-9a6d-4371-adc3-d0e65dee6826-kube-api-access-cc87s\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 01:56:42.213458 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.213423 2580 generic.go:356] "Generic (PLEG): container finished" podID="b8cd3c46-9a6d-4371-adc3-d0e65dee6826" containerID="1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0" exitCode=0 Jul 18 01:56:42.213876 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.213473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f887f797d-7lhxd" event={"ID":"b8cd3c46-9a6d-4371-adc3-d0e65dee6826","Type":"ContainerDied","Data":"1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0"} Jul 18 01:56:42.213876 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.213485 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f887f797d-7lhxd" Jul 18 01:56:42.213876 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.213512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f887f797d-7lhxd" event={"ID":"b8cd3c46-9a6d-4371-adc3-d0e65dee6826","Type":"ContainerDied","Data":"730a80b48e04efafcefabb142cbb0233a9ce64430cd8f138238a219b1e9b1a45"} Jul 18 01:56:42.213876 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.213532 2580 scope.go:117] "RemoveContainer" containerID="1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0" Jul 18 01:56:42.222515 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.222498 2580 scope.go:117] "RemoveContainer" containerID="1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0" Jul 18 01:56:42.222779 ip-10-0-137-56 kubenswrapper[2580]: E0718 01:56:42.222760 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0\": container with ID starting with 1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0 not found: ID does not exist" containerID="1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0" Jul 18 01:56:42.222828 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.222787 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0"} err="failed to get container status \"1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0\": rpc error: code = NotFound desc = could not find container \"1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0\": container with ID starting with 1947de83426c163032c98c9766ac519bb17e708d2c121f5586de31557318b9f0 not found: ID does not exist" Jul 18 01:56:42.236199 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.236176 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f887f797d-7lhxd"] Jul 18 01:56:42.242794 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.242772 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f887f797d-7lhxd"] Jul 18 01:56:42.897698 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:56:42.897665 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8cd3c46-9a6d-4371-adc3-d0e65dee6826" path="/var/lib/kubelet/pods/b8cd3c46-9a6d-4371-adc3-d0e65dee6826/volumes" Jul 18 01:57:00.295929 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.295891 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:57:00.296330 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.296291 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8cd3c46-9a6d-4371-adc3-d0e65dee6826" containerName="console" Jul 18 01:57:00.296330 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.296306 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8cd3c46-9a6d-4371-adc3-d0e65dee6826" containerName="console" Jul 18 01:57:00.296403 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.296361 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8cd3c46-9a6d-4371-adc3-d0e65dee6826" containerName="console" Jul 18 01:57:00.300859 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.300840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.302863 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.302843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 18 01:57:00.302986 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.302928 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 01:57:00.304119 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.304088 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 01:57:00.304254 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.304159 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qrwf8\"" Jul 18 01:57:00.307109 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.307065 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:57:00.332836 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.332814 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:57:00.361911 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.361887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41c8968c-4630-4aa9-8a4f-6ebba96dbc5d-config-file\") pod \"limitador-limitador-54d6d988d5-clrdc\" (UID: \"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.362026 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.361927 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4knm4\" (UniqueName: \"kubernetes.io/projected/41c8968c-4630-4aa9-8a4f-6ebba96dbc5d-kube-api-access-4knm4\") pod \"limitador-limitador-54d6d988d5-clrdc\" (UID: \"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.462598 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.462546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4knm4\" (UniqueName: \"kubernetes.io/projected/41c8968c-4630-4aa9-8a4f-6ebba96dbc5d-kube-api-access-4knm4\") pod \"limitador-limitador-54d6d988d5-clrdc\" (UID: \"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.462729 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.462685 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41c8968c-4630-4aa9-8a4f-6ebba96dbc5d-config-file\") pod \"limitador-limitador-54d6d988d5-clrdc\" (UID: \"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.463232 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.463214 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/41c8968c-4630-4aa9-8a4f-6ebba96dbc5d-config-file\") pod \"limitador-limitador-54d6d988d5-clrdc\" (UID: \"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.473291 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.473265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4knm4\" (UniqueName: \"kubernetes.io/projected/41c8968c-4630-4aa9-8a4f-6ebba96dbc5d-kube-api-access-4knm4\") pod \"limitador-limitador-54d6d988d5-clrdc\" (UID: \"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.612284 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.612190 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:00.739207 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:00.739182 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:57:00.741438 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:57:00.741398 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c8968c_4630_4aa9_8a4f_6ebba96dbc5d.slice/crio-e0491574dee524dc78fcfa00a3a96aacb4932c6dfeec3e585a1e8387869567b7 WatchSource:0}: Error finding container e0491574dee524dc78fcfa00a3a96aacb4932c6dfeec3e585a1e8387869567b7: Status 404 returned error can't find the container with id e0491574dee524dc78fcfa00a3a96aacb4932c6dfeec3e585a1e8387869567b7 Jul 18 01:57:01.311226 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:01.311133 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" event={"ID":"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d","Type":"ContainerStarted","Data":"e0491574dee524dc78fcfa00a3a96aacb4932c6dfeec3e585a1e8387869567b7"} Jul 18 01:57:03.324388 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:03.324350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" event={"ID":"41c8968c-4630-4aa9-8a4f-6ebba96dbc5d","Type":"ContainerStarted","Data":"9f81fb5a1e1a42e7bb568058814fb70f2967c0ef0a0aa7315cc324889d755a0c"} Jul 18 01:57:03.324738 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:03.324414 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:57:03.342653 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:03.342603 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" podStartSLOduration=0.878241772 podStartE2EDuration="3.342575006s" totalImagesPullingTime="2.464333234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:03.339567516 +0000 UTC m=+529.023440492" watchObservedRunningTime="2026-07-18 01:57:03.342575006 +0000 UTC m=+529.026447761" Jul 18 01:57:14.331109 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:57:14.331028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-clrdc" Jul 18 01:58:11.507458 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:58:11.507422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:58:14.770968 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:58:14.770939 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:58:14.775274 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:58:14.775254 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 01:58:28.899192 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:58:28.899159 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:58:39.119076 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:58:39.119001 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:58:50.998725 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:58:50.998691 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:59:01.495916 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:01.495879 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:59:10.098643 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.098611 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq"] Jul 18 01:59:10.101906 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.101886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.104782 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.104758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mb98f\"" Jul 18 01:59:10.104879 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.104763 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 18 01:59:10.104879 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.104808 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 01:59:10.104879 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.104856 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 01:59:10.110290 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.110270 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq"] Jul 18 01:59:10.139251 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.139230 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31aeab0-beba-4dcf-a184-822682cda2bd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.139357 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.139257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.139357 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.139285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.139357 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.139323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.139357 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.139338 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9ng5\" (UniqueName: \"kubernetes.io/projected/e31aeab0-beba-4dcf-a184-822682cda2bd-kube-api-access-z9ng5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.139357 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.139355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240466 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240560 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9ng5\" (UniqueName: \"kubernetes.io/projected/e31aeab0-beba-4dcf-a184-822682cda2bd-kube-api-access-z9ng5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240560 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240662 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31aeab0-beba-4dcf-a184-822682cda2bd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240662 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240741 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240804 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240789 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.240975 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.240958 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.241070 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.241052 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.242750 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.242735 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e31aeab0-beba-4dcf-a184-822682cda2bd-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.242871 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.242856 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e31aeab0-beba-4dcf-a184-822682cda2bd-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.247897 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.247879 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9ng5\" (UniqueName: \"kubernetes.io/projected/e31aeab0-beba-4dcf-a184-822682cda2bd-kube-api-access-z9ng5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-st5lq\" (UID: \"e31aeab0-beba-4dcf-a184-822682cda2bd\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.413378 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.413307 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:10.539934 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.537501 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq"] Jul 18 01:59:10.540654 ip-10-0-137-56 kubenswrapper[2580]: W0718 01:59:10.540625 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31aeab0_beba_4dcf_a184_822682cda2bd.slice/crio-0eeb81353641e22d65f26dfd31ae512e93ff910cf2704118eccb4740b8c04708 WatchSource:0}: Error finding container 0eeb81353641e22d65f26dfd31ae512e93ff910cf2704118eccb4740b8c04708: Status 404 returned error can't find the container with id 0eeb81353641e22d65f26dfd31ae512e93ff910cf2704118eccb4740b8c04708 Jul 18 01:59:10.542121 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.542100 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:59:10.921740 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:10.921709 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" event={"ID":"e31aeab0-beba-4dcf-a184-822682cda2bd","Type":"ContainerStarted","Data":"0eeb81353641e22d65f26dfd31ae512e93ff910cf2704118eccb4740b8c04708"} Jul 18 01:59:11.927798 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:11.927716 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" event={"ID":"e31aeab0-beba-4dcf-a184-822682cda2bd","Type":"ContainerStarted","Data":"a37668fca135b958ff59898c3df14d48280b7dd08eb0837fb8b8adb6295861e0"} Jul 18 01:59:11.945544 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:11.945497 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" podStartSLOduration=0.967458069 podStartE2EDuration="1.945483715s" totalImagesPullingTime="978.025646ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:11.944089796 +0000 UTC m=+657.627962535" watchObservedRunningTime="2026-07-18 01:59:11.945483715 +0000 UTC m=+657.629356452" Jul 18 01:59:12.200601 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:12.200504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:59:19.809988 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:19.809952 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 01:59:20.414090 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:20.414056 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:20.414280 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:20.414109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:20.426344 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:20.426322 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 01:59:20.979359 ip-10-0-137-56 kubenswrapper[2580]: I0718 01:59:20.979332 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-st5lq" Jul 18 02:00:53.398600 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:00:53.398553 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:01:03.208251 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:01:03.208217 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:01:13.903197 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:01:13.903158 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:02:57.207406 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:02:57.207364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:03:07.805602 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:03:07.805504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:03:14.806376 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:03:14.806342 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:03:14.808825 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:03:14.808797 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:03:40.195948 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:03:40.195901 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:03:50.407517 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:03:50.407482 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:03:58.911070 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:03:58.911032 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:04:08.201994 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:04:08.201955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:04:18.798039 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:04:18.797994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:04:30.705794 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:04:30.705755 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:04:41.303089 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:04:41.303006 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:04:50.796928 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:04:50.796880 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:05:16.604755 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:05:16.604709 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:05:25.906660 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:05:25.906620 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:05:50.005391 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:05:50.005350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:06:00.597548 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:06:00.597507 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:06:15.201828 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:06:15.201745 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:06:25.806878 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:06:25.806832 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:06:35.709753 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:06:35.709708 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:06:45.698431 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:06:45.698390 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:06:59.797086 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:06:59.797051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:08:14.843636 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:08:14.843600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:08:14.844240 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:08:14.843740 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:08:44.304419 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:08:44.304376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:08:53.098030 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:08:53.097996 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:09:05.901498 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:09:05.901456 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:09:19.709062 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:09:19.708978 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:09:29.199019 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:09:29.198981 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:09:38.696407 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:09:38.696358 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:09:47.691720 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:09:47.691677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:09:58.799115 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:09:58.799076 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:10:10.700925 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:10:10.700886 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:10:18.903464 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:10:18.903425 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:10:29.706337 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:10:29.706301 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:10:53.797633 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:10:53.797527 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:11:01.500098 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:11:01.500053 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:11:11.601421 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:11:11.601385 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:11:21.206307 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:11:21.206269 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:11:33.700805 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:11:33.700773 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:11:49.105646 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:11:49.105605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:11:59.796371 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:11:59.796326 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:12:09.499420 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:12:09.499219 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:12:21.200692 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:12:21.200658 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:12:30.410414 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:12:30.410369 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:12:39.901571 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:12:39.901536 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:12:52.098725 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:12:52.098678 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:13:01.499076 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:01.499033 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:13:10.903050 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:10.903007 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:13:14.873808 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:14.873779 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:13:14.875215 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:14.875193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:13:20.705947 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:20.705904 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:13:34.100081 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:34.100041 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:13:40.996987 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:40.996948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:13:57.542791 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:13:57.542755 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:14:09.099145 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:09.099111 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:14:20.300523 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:20.300488 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:14:29.497761 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:29.497725 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:14:52.197826 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:52.197789 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:14:56.140224 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.140191 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7"] Jul 18 02:14:56.143575 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.143553 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.145809 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.145788 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c62b43cb-openshift-default-dockercfg-cfg98\"" Jul 18 02:14:56.171673 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.171638 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7"] Jul 18 02:14:56.227043 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-token\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227268 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227069 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-certs\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227268 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-podinfo\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227268 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5296e045-fe5f-41d8-aca8-408751d60ce9-istiod-ca-cert\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227268 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227216 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-socket\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227268 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-data\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227535 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h57k7\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-kube-api-access-h57k7\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227535 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-credential-socket\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.227535 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.227398 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-envoy\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.328482 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.328431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h57k7\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-kube-api-access-h57k7\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.328710 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.328493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-credential-socket\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.328710 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.328523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-envoy\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.328710 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.328608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-token\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.328710 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.328652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-certs\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.328710 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.328689 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-podinfo\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329052 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-credential-socket\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329163 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329068 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-certs\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329163 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5296e045-fe5f-41d8-aca8-408751d60ce9-istiod-ca-cert\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329261 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329179 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-socket\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329261 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-data\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329516 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329476 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-socket\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329601 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329515 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-data\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.329675 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.329653 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5296e045-fe5f-41d8-aca8-408751d60ce9-istiod-ca-cert\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.330902 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.330873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-envoy\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.331375 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.331351 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-podinfo\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.337370 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.337335 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-token\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.337502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.337408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h57k7\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-kube-api-access-h57k7\") pod \"e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.454882 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.454781 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:56.597726 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.597695 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7"] Jul 18 02:14:56.599830 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:14:56.599798 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5296e045_fe5f_41d8_aca8_408751d60ce9.slice/crio-2843433aee99626d96e67dcd5d62e61fd1adc45fe9f142d55832d41db1bdbb0d WatchSource:0}: Error finding container 2843433aee99626d96e67dcd5d62e61fd1adc45fe9f142d55832d41db1bdbb0d: Status 404 returned error can't find the container with id 2843433aee99626d96e67dcd5d62e61fd1adc45fe9f142d55832d41db1bdbb0d Jul 18 02:14:56.601830 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.601796 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:14:56.601932 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.601884 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:14:56.601992 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:56.601929 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:14:57.554373 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:57.554335 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" event={"ID":"5296e045-fe5f-41d8-aca8-408751d60ce9","Type":"ContainerStarted","Data":"c30a1503d820f2c0d9bd26b219de96448557b63d6091dfca0ff4be869f8952ff"} Jul 18 02:14:57.554899 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:57.554394 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" event={"ID":"5296e045-fe5f-41d8-aca8-408751d60ce9","Type":"ContainerStarted","Data":"2843433aee99626d96e67dcd5d62e61fd1adc45fe9f142d55832d41db1bdbb0d"} Jul 18 02:14:57.572535 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:57.572480 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podStartSLOduration=1.572462315 podStartE2EDuration="1.572462315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:14:57.571016546 +0000 UTC m=+1603.254889287" watchObservedRunningTime="2026-07-18 02:14:57.572462315 +0000 UTC m=+1603.256335099" Jul 18 02:14:58.456000 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:58.455960 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:58.461928 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:58.461898 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:58.560118 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:58.560086 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:14:58.561163 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:14:58.561137 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:05.400877 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:05.400834 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:15:13.561970 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.561932 2580 patch_prober.go:28] interesting pod/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 18 02:15:13.562399 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.561990 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 18 02:15:13.640884 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.640850 2580 generic.go:356] "Generic (PLEG): container finished" podID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerID="c30a1503d820f2c0d9bd26b219de96448557b63d6091dfca0ff4be869f8952ff" exitCode=0 Jul 18 02:15:13.641071 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.640920 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" event={"ID":"5296e045-fe5f-41d8-aca8-408751d60ce9","Type":"ContainerDied","Data":"c30a1503d820f2c0d9bd26b219de96448557b63d6091dfca0ff4be869f8952ff"} Jul 18 02:15:13.641328 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.641310 2580 scope.go:117] "RemoveContainer" containerID="c30a1503d820f2c0d9bd26b219de96448557b63d6091dfca0ff4be869f8952ff" Jul 18 02:15:13.642287 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.642259 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:15:13.642340 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.642327 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:15:13.642374 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:13.642360 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:15:14.653186 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:14.652257 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" event={"ID":"5296e045-fe5f-41d8-aca8-408751d60ce9","Type":"ContainerStarted","Data":"0eeb2d7ebbd0cf52c79f84d2db4d32174c4cec50d80007269d4db8b2375454f9"} Jul 18 02:15:15.456139 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:15.456097 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:15.456642 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:15.456471 2580 patch_prober.go:28] interesting pod/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 18 02:15:15.456851 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:15.456826 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 18 02:15:16.455632 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:16.455568 2580 patch_prober.go:28] interesting pod/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 18 02:15:16.456275 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:16.456240 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 18 02:15:17.456031 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:17.455988 2580 patch_prober.go:28] interesting pod/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 18 02:15:17.456649 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:17.456613 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 18 02:15:18.456021 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:18.455977 2580 patch_prober.go:28] interesting pod/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Jul 18 02:15:18.456248 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:18.456054 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Jul 18 02:15:19.459347 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:19.459315 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:19.459765 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:19.459560 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:19.460359 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:19.460344 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:37.602301 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:37.602269 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:15:39.639203 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:39.639170 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7"] Jul 18 02:15:39.639624 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:39.639386 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" containerID="cri-o://0eeb2d7ebbd0cf52c79f84d2db4d32174c4cec50d80007269d4db8b2375454f9" gracePeriod=30 Jul 18 02:15:44.806999 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.806963 2580 generic.go:356] "Generic (PLEG): container finished" podID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerID="0eeb2d7ebbd0cf52c79f84d2db4d32174c4cec50d80007269d4db8b2375454f9" exitCode=0 Jul 18 02:15:44.807445 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.807035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" event={"ID":"5296e045-fe5f-41d8-aca8-408751d60ce9","Type":"ContainerDied","Data":"0eeb2d7ebbd0cf52c79f84d2db4d32174c4cec50d80007269d4db8b2375454f9"} Jul 18 02:15:44.807445 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.807099 2580 scope.go:117] "RemoveContainer" containerID="c30a1503d820f2c0d9bd26b219de96448557b63d6091dfca0ff4be869f8952ff" Jul 18 02:15:44.902870 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.902792 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:44.943472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943436 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5296e045-fe5f-41d8-aca8-408751d60ce9-istiod-ca-cert\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943491 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-podinfo\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943521 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-token\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-credential-socket\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-envoy\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943725 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-certs\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943995 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943767 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-data\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943995 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943805 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h57k7\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-kube-api-access-h57k7\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943995 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943854 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-socket\") pod \"5296e045-fe5f-41d8-aca8-408751d60ce9\" (UID: \"5296e045-fe5f-41d8-aca8-408751d60ce9\") " Jul 18 02:15:44.943995 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943892 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:44.943995 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943973 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5296e045-fe5f-41d8-aca8-408751d60ce9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:15:44.944247 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.943998 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:44.944247 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.944201 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:44.944345 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.944324 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:44.944400 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.944344 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-workload-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:44.944400 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.944358 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5296e045-fe5f-41d8-aca8-408751d60ce9-istiod-ca-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:44.944400 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.944373 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-credential-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:44.944773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.944743 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-data" (OuterVolumeSpecName: "istio-data") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:44.946173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.946140 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-kube-api-access-h57k7" (OuterVolumeSpecName: "kube-api-access-h57k7") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "kube-api-access-h57k7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:15:44.946173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.946152 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:44.946311 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.946251 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:15:44.946311 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:44.946270 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-token" (OuterVolumeSpecName: "istio-token") pod "5296e045-fe5f-41d8-aca8-408751d60ce9" (UID: "5296e045-fe5f-41d8-aca8-408751d60ce9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:15:45.045020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.044985 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-podinfo\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:45.045020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.045015 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-token\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:45.045020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.045024 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-envoy\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:45.045020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.045032 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5296e045-fe5f-41d8-aca8-408751d60ce9-istio-data\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:45.045292 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.045041 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h57k7\" (UniqueName: \"kubernetes.io/projected/5296e045-fe5f-41d8-aca8-408751d60ce9-kube-api-access-h57k7\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:15:45.814010 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.813981 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" Jul 18 02:15:45.814434 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.814033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7" event={"ID":"5296e045-fe5f-41d8-aca8-408751d60ce9","Type":"ContainerDied","Data":"2843433aee99626d96e67dcd5d62e61fd1adc45fe9f142d55832d41db1bdbb0d"} Jul 18 02:15:45.814434 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.814089 2580 scope.go:117] "RemoveContainer" containerID="0eeb2d7ebbd0cf52c79f84d2db4d32174c4cec50d80007269d4db8b2375454f9" Jul 18 02:15:45.836162 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.836115 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7"] Jul 18 02:15:45.839915 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:45.839882 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c62b43cb-openshift-default-7bc69687c-f4dz7"] Jul 18 02:15:46.897502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:46.897466 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" path="/var/lib/kubelet/pods/5296e045-fe5f-41d8-aca8-408751d60ce9/volumes" Jul 18 02:15:50.005087 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:15:50.005048 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:16:21.596960 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:21.596921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:16:24.373429 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.373397 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx"] Jul 18 02:16:24.373913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.373763 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" Jul 18 02:16:24.373913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.373776 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" Jul 18 02:16:24.373913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.373843 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" Jul 18 02:16:24.373913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.373852 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" Jul 18 02:16:24.376632 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.376613 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.378681 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.378660 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9a88968c-openshift-default-dockercfg-6hwpw\"" Jul 18 02:16:24.389107 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.389083 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx"] Jul 18 02:16:24.476984 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.476952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-data\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477149 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.476993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-socket\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477149 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-certs\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477149 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-token\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477149 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-envoy\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477302 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdbnm\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-kube-api-access-vdbnm\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477302 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9246e8-b768-4ea5-878a-8dfbae13f508-istiod-ca-cert\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477302 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-credential-socket\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.477302 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.477282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-podinfo\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578058 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-credential-socket\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578058 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-podinfo\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-data\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-socket\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-certs\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578227 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-token\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-envoy\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdbnm\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-kube-api-access-vdbnm\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9246e8-b768-4ea5-878a-8dfbae13f508-istiod-ca-cert\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578495 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-data\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578572 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-certs\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578802 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578739 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-socket\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.578861 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.578821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-credential-socket\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.579102 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.579082 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9246e8-b768-4ea5-878a-8dfbae13f508-istiod-ca-cert\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.580607 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.580572 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-envoy\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.580938 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.580922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-podinfo\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.586673 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.586649 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-token\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.587114 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.587095 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdbnm\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-kube-api-access-vdbnm\") pod \"e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.689869 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.689779 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:24.825869 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.825846 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx"] Jul 18 02:16:24.827864 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:16:24.827832 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9246e8_b768_4ea5_878a_8dfbae13f508.slice/crio-acf35b9f154e3b49493897fdb170742dafbfa4e72ce7f744a70e807556812293 WatchSource:0}: Error finding container acf35b9f154e3b49493897fdb170742dafbfa4e72ce7f744a70e807556812293: Status 404 returned error can't find the container with id acf35b9f154e3b49493897fdb170742dafbfa4e72ce7f744a70e807556812293 Jul 18 02:16:24.830017 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.829983 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:16:24.830109 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.830049 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:16:24.830109 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:24.830078 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:16:25.005651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:25.005615 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" event={"ID":"ff9246e8-b768-4ea5-878a-8dfbae13f508","Type":"ContainerStarted","Data":"566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9"} Jul 18 02:16:25.005824 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:25.005670 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" event={"ID":"ff9246e8-b768-4ea5-878a-8dfbae13f508","Type":"ContainerStarted","Data":"acf35b9f154e3b49493897fdb170742dafbfa4e72ce7f744a70e807556812293"} Jul 18 02:16:25.026672 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:25.026611 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podStartSLOduration=1.026592666 podStartE2EDuration="1.026592666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:16:25.023641628 +0000 UTC m=+1690.707514368" watchObservedRunningTime="2026-07-18 02:16:25.026592666 +0000 UTC m=+1690.710465396" Jul 18 02:16:25.690276 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:25.690239 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:25.694684 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:25.694660 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:26.010953 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:26.010923 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:26.012012 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:26.011989 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:34.101539 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:34.101496 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:16:41.469201 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:41.468785 2580 patch_prober.go:28] interesting pod/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:52956->10.133.0.40:15021: read: connection reset by peer" start-of-body= Jul 18 02:16:41.469201 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:41.468887 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": read tcp 10.133.0.2:52956->10.133.0.40:15021: read: connection reset by peer" Jul 18 02:16:42.098865 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:42.098829 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerID="566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9" exitCode=0 Jul 18 02:16:42.099041 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:42.098898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" event={"ID":"ff9246e8-b768-4ea5-878a-8dfbae13f508","Type":"ContainerDied","Data":"566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9"} Jul 18 02:16:42.099412 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:42.099387 2580 scope.go:117] "RemoveContainer" containerID="566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9" Jul 18 02:16:42.100405 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:42.100377 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:16:42.100504 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:42.100460 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:16:42.100570 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:42.100507 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:16:43.105737 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:43.105686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" event={"ID":"ff9246e8-b768-4ea5-878a-8dfbae13f508","Type":"ContainerStarted","Data":"b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa"} Jul 18 02:16:43.689986 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:43.689945 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:43.690208 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:43.690185 2580 patch_prober.go:28] interesting pod/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 18 02:16:43.690256 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:43.690238 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 18 02:16:44.690499 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:44.690464 2580 patch_prober.go:28] interesting pod/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 18 02:16:44.690922 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:44.690519 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 18 02:16:45.690690 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:45.690649 2580 patch_prober.go:28] interesting pod/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 18 02:16:45.691098 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:45.690714 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 18 02:16:46.690685 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:46.690645 2580 patch_prober.go:28] interesting pod/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 18 02:16:46.690860 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:46.690703 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 18 02:16:47.693734 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:47.693703 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:47.694170 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:47.693973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:16:47.694672 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:16:47.694653 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:17:05.500098 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:05.500057 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:17:08.413890 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:08.413852 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx"] Jul 18 02:17:08.414264 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:08.414107 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" containerID="cri-o://b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa" gracePeriod=30 Jul 18 02:17:13.663342 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.663313 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:17:13.796494 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796457 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-socket\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796494 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796497 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-podinfo\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796744 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796531 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-credential-socket\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796744 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796561 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9246e8-b768-4ea5-878a-8dfbae13f508-istiod-ca-cert\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796744 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796728 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdbnm\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-kube-api-access-vdbnm\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796920 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796787 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-envoy\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796920 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796827 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-data\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796920 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796878 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-certs\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.796920 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796888 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:13.797123 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796932 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-token\") pod \"ff9246e8-b768-4ea5-878a-8dfbae13f508\" (UID: \"ff9246e8-b768-4ea5-878a-8dfbae13f508\") " Jul 18 02:17:13.797123 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.796945 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:13.797123 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.797022 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff9246e8-b768-4ea5-878a-8dfbae13f508-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:17:13.797269 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.797218 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.797269 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.797236 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-credential-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.797269 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.797250 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff9246e8-b768-4ea5-878a-8dfbae13f508-istiod-ca-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.797408 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.797273 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:13.797852 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.797830 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:13.798963 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.798934 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-kube-api-access-vdbnm" (OuterVolumeSpecName: "kube-api-access-vdbnm") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "kube-api-access-vdbnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:17:13.799196 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.799170 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:17:13.799322 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.799299 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:13.799373 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.799303 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff9246e8-b768-4ea5-878a-8dfbae13f508" (UID: "ff9246e8-b768-4ea5-878a-8dfbae13f508"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:17:13.898564 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.898517 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdbnm\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-kube-api-access-vdbnm\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.898564 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.898554 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-envoy\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.898564 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.898566 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-data\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.898564 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.898576 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff9246e8-b768-4ea5-878a-8dfbae13f508-workload-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.898564 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.898601 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-token\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:13.898884 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:13.898609 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff9246e8-b768-4ea5-878a-8dfbae13f508-istio-podinfo\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:17:14.260124 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.260089 2580 generic.go:356] "Generic (PLEG): container finished" podID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerID="b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa" exitCode=0 Jul 18 02:17:14.260317 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.260141 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" Jul 18 02:17:14.260317 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.260190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" event={"ID":"ff9246e8-b768-4ea5-878a-8dfbae13f508","Type":"ContainerDied","Data":"b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa"} Jul 18 02:17:14.260317 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.260248 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx" event={"ID":"ff9246e8-b768-4ea5-878a-8dfbae13f508","Type":"ContainerDied","Data":"acf35b9f154e3b49493897fdb170742dafbfa4e72ce7f744a70e807556812293"} Jul 18 02:17:14.260317 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.260280 2580 scope.go:117] "RemoveContainer" containerID="b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa" Jul 18 02:17:14.269686 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.269662 2580 scope.go:117] "RemoveContainer" containerID="566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9" Jul 18 02:17:14.278273 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.278250 2580 scope.go:117] "RemoveContainer" containerID="b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa" Jul 18 02:17:14.278704 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:17:14.278571 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa\": container with ID starting with b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa not found: ID does not exist" containerID="b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa" Jul 18 02:17:14.278704 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.278662 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa"} err="failed to get container status \"b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa\": rpc error: code = NotFound desc = could not find container \"b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa\": container with ID starting with b1c7a7023a03d69ff364749c502cbda5731e536e09e31586de2db0f23ceecafa not found: ID does not exist" Jul 18 02:17:14.278704 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.278687 2580 scope.go:117] "RemoveContainer" containerID="566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9" Jul 18 02:17:14.279095 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:17:14.279070 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9\": container with ID starting with 566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9 not found: ID does not exist" containerID="566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9" Jul 18 02:17:14.279187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.279103 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9"} err="failed to get container status \"566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9\": rpc error: code = NotFound desc = could not find container \"566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9\": container with ID starting with 566894b5f573cac92849d7ce1a7283175d7994d2aeb76e1f30557127e77d57f9 not found: ID does not exist" Jul 18 02:17:14.281450 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.281426 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx"] Jul 18 02:17:14.285459 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.285432 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9a88968c-openshift-default-7c968c8695-6thzx"] Jul 18 02:17:14.897755 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:14.897715 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" path="/var/lib/kubelet/pods/ff9246e8-b768-4ea5-878a-8dfbae13f508/volumes" Jul 18 02:17:17.999944 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:17.999903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:17:49.605457 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:17:49.605420 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:18:14.907473 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:18:14.907441 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:18:14.909854 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:18:14.909828 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:19:18.806876 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:19:18.806830 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:19:28.297376 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:19:28.297336 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:19:38.503419 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:19:38.503325 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:19:59.008136 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:19:59.008101 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:20:34.902832 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:20:34.902791 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:20:45.316438 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:20:45.316388 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:21:00.792220 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:21:00.792181 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:21:17.099418 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:21:17.099324 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:21:27.193839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:21:27.193800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:22:25.195877 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:22:25.195836 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:22:33.130406 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:22:33.130367 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:23:14.938084 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:14.938051 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:23:14.941347 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:14.941326 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:23:18.363709 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.363672 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x"] Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364028 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364040 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364053 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364059 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364069 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364075 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5296e045-fe5f-41d8-aca8-408751d60ce9" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364126 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" Jul 18 02:23:18.364187 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.364133 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9246e8-b768-4ea5-878a-8dfbae13f508" containerName="istio-proxy" Jul 18 02:23:18.367043 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.367020 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.369048 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.369025 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f21b76-openshift-default-dockercfg-75zz7\"" Jul 18 02:23:18.379637 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.379567 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x"] Jul 18 02:23:18.532923 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.532881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc24e0a-00b7-42d3-9532-49b43095b183-istiod-ca-cert\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.532969 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-envoy\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.532999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-credential-socket\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.533029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-certs\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.533052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-podinfo\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.533071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89hb2\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-kube-api-access-89hb2\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.533099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-token\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.533124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-data\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.533428 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.533195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-socket\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-credential-socket\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634083 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-certs\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634104 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-podinfo\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89hb2\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-kube-api-access-89hb2\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-token\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-data\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-socket\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634398 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc24e0a-00b7-42d3-9532-49b43095b183-istiod-ca-cert\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-envoy\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634519 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-credential-socket\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-data\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634675 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-certs\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.634839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.634727 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-socket\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.635211 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.635186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc24e0a-00b7-42d3-9532-49b43095b183-istiod-ca-cert\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.636596 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.636567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-envoy\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.636811 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.636793 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-podinfo\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.644270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.644247 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-token\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.644424 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.644402 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89hb2\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-kube-api-access-89hb2\") pod \"e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.678336 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.678293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:18.817641 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.817613 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x"] Jul 18 02:23:18.819505 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:23:18.819480 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc24e0a_00b7_42d3_9532_49b43095b183.slice/crio-d7b5b8e1acbd995992c16889dde9bb2197fffc52b49e072815839af04478d725 WatchSource:0}: Error finding container d7b5b8e1acbd995992c16889dde9bb2197fffc52b49e072815839af04478d725: Status 404 returned error can't find the container with id d7b5b8e1acbd995992c16889dde9bb2197fffc52b49e072815839af04478d725 Jul 18 02:23:18.821781 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.821747 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:23:18.821903 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.821813 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:23:18.821903 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:18.821843 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:23:19.037498 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:19.037461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" event={"ID":"4dc24e0a-00b7-42d3-9532-49b43095b183","Type":"ContainerStarted","Data":"463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548"} Jul 18 02:23:19.037696 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:19.037515 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" event={"ID":"4dc24e0a-00b7-42d3-9532-49b43095b183","Type":"ContainerStarted","Data":"d7b5b8e1acbd995992c16889dde9bb2197fffc52b49e072815839af04478d725"} Jul 18 02:23:19.062351 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:19.062292 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podStartSLOduration=1.062277142 podStartE2EDuration="1.062277142s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:19.059622189 +0000 UTC m=+2104.743494924" watchObservedRunningTime="2026-07-18 02:23:19.062277142 +0000 UTC m=+2104.746149879" Jul 18 02:23:19.678708 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:19.678671 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:19.683746 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:19.683718 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:20.042643 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:20.042616 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:20.043605 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:20.043561 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:20.701907 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:20.701867 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:23:34.802067 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:34.802023 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:23:36.085790 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:36.085745 2580 patch_prober.go:28] interesting pod/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 02:23:36.086303 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:36.085811 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 02:23:37.132854 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:37.132763 2580 generic.go:356] "Generic (PLEG): container finished" podID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerID="463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548" exitCode=0 Jul 18 02:23:37.132854 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:37.132816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" event={"ID":"4dc24e0a-00b7-42d3-9532-49b43095b183","Type":"ContainerDied","Data":"463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548"} Jul 18 02:23:37.133294 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:37.133185 2580 scope.go:117] "RemoveContainer" containerID="463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548" Jul 18 02:23:37.134292 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:37.134225 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:23:37.134368 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:37.134352 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:23:37.134406 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:37.134393 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:23:38.139870 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:38.139835 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" event={"ID":"4dc24e0a-00b7-42d3-9532-49b43095b183","Type":"ContainerStarted","Data":"85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a"} Jul 18 02:23:38.678659 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:38.678619 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:38.678887 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:38.678863 2580 patch_prober.go:28] interesting pod/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 18 02:23:38.678935 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:38.678914 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 18 02:23:39.679549 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:39.679503 2580 patch_prober.go:28] interesting pod/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 18 02:23:39.679952 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:39.679574 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 18 02:23:40.679183 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:40.679138 2580 patch_prober.go:28] interesting pod/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 18 02:23:40.679361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:40.679215 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 18 02:23:41.679240 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:41.679200 2580 patch_prober.go:28] interesting pod/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 18 02:23:41.679665 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:41.679270 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 18 02:23:42.682774 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:42.682738 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:42.683177 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:42.683010 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:42.683937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:42.683915 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:23:46.703517 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:23:46.703473 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:24:13.012376 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.012272 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v"] Jul 18 02:24:13.014960 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.014938 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.018021 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.017987 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f21b76\"/\"kube-root-ca.crt\"" Jul 18 02:24:13.018021 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.017982 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f21b76\"/\"openshift-service-ca.crt\"" Jul 18 02:24:13.018250 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.017991 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f21b76\"/\"default-dockercfg-zdsxb\"" Jul 18 02:24:13.018250 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.018047 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f21b76\"/\"test-model-f21b76-kserve-self-signed-certs\"" Jul 18 02:24:13.025957 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.025927 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v"] Jul 18 02:24:13.154467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.154431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-model-cache\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.154666 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.154485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tls-certs\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.154666 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.154560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tmp-dir\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.154666 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.154630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-dshm\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.154666 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.154666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-home\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.154820 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.154691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9gnc\" (UniqueName: \"kubernetes.io/projected/f53d405d-1548-4f19-9dae-4f7abeab0c7b-kube-api-access-f9gnc\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.255634 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.255573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-dshm\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.255834 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.255662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-home\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.255834 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.255693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9gnc\" (UniqueName: \"kubernetes.io/projected/f53d405d-1548-4f19-9dae-4f7abeab0c7b-kube-api-access-f9gnc\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.255834 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.255714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-model-cache\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.255834 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.255763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tls-certs\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.255834 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.255805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tmp-dir\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.256173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.256132 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-home\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.256173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.256148 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tmp-dir\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.256299 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.256214 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-model-cache\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.258019 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.257991 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-dshm\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.258371 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.258348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tls-certs\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.265110 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.265021 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9gnc\" (UniqueName: \"kubernetes.io/projected/f53d405d-1548-4f19-9dae-4f7abeab0c7b-kube-api-access-f9gnc\") pod \"test-model-f21b76-kserve-6b7795fcbf-hdr9v\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.324658 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.324620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:13.466635 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:13.466605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v"] Jul 18 02:24:13.468552 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:24:13.468524 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf53d405d_1548_4f19_9dae_4f7abeab0c7b.slice/crio-540ff7ae07f6531385606671b2ae40cb3f594f83b6c35264aff38ba58dcd877c WatchSource:0}: Error finding container 540ff7ae07f6531385606671b2ae40cb3f594f83b6c35264aff38ba58dcd877c: Status 404 returned error can't find the container with id 540ff7ae07f6531385606671b2ae40cb3f594f83b6c35264aff38ba58dcd877c Jul 18 02:24:14.335258 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:14.335217 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" event={"ID":"f53d405d-1548-4f19-9dae-4f7abeab0c7b","Type":"ContainerStarted","Data":"28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24"} Jul 18 02:24:14.335258 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:14.335262 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" event={"ID":"f53d405d-1548-4f19-9dae-4f7abeab0c7b","Type":"ContainerStarted","Data":"540ff7ae07f6531385606671b2ae40cb3f594f83b6c35264aff38ba58dcd877c"} Jul 18 02:24:14.351260 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:14.351198 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" podStartSLOduration=2.35117878 podStartE2EDuration="2.35117878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:24:14.350050789 +0000 UTC m=+2160.033923540" watchObservedRunningTime="2026-07-18 02:24:14.35117878 +0000 UTC m=+2160.035051518" Jul 18 02:24:17.504932 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:17.504874 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:24:23.325121 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:23.325062 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:23.325757 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:23.325139 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:23.337940 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:23.337909 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:23.391065 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:23.391032 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:32.128063 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:32.128025 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:24:40.412046 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:40.412000 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v"] Jul 18 02:24:44.504044 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:44.504003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:24:54.403691 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:54.403651 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:24:55.433302 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.433245 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" podUID="f53d405d-1548-4f19-9dae-4f7abeab0c7b" containerName="main" containerID="cri-o://28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24" gracePeriod=45 Jul 18 02:24:55.688305 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.688244 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:55.836263 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836206 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tmp-dir\") pod \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " Jul 18 02:24:55.836263 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836264 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9gnc\" (UniqueName: \"kubernetes.io/projected/f53d405d-1548-4f19-9dae-4f7abeab0c7b-kube-api-access-f9gnc\") pod \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " Jul 18 02:24:55.836540 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836331 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-model-cache\") pod \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " Jul 18 02:24:55.836540 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836355 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tls-certs\") pod \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " Jul 18 02:24:55.836540 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836380 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-home\") pod \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " Jul 18 02:24:55.836540 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836407 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-dshm\") pod \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\" (UID: \"f53d405d-1548-4f19-9dae-4f7abeab0c7b\") " Jul 18 02:24:55.836796 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836571 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f53d405d-1548-4f19-9dae-4f7abeab0c7b" (UID: "f53d405d-1548-4f19-9dae-4f7abeab0c7b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:55.836796 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836659 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-model-cache" (OuterVolumeSpecName: "model-cache") pod "f53d405d-1548-4f19-9dae-4f7abeab0c7b" (UID: "f53d405d-1548-4f19-9dae-4f7abeab0c7b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:55.836796 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836681 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-home" (OuterVolumeSpecName: "home") pod "f53d405d-1548-4f19-9dae-4f7abeab0c7b" (UID: "f53d405d-1548-4f19-9dae-4f7abeab0c7b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:55.836796 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.836718 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tmp-dir\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:24:55.838709 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.838677 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-dshm" (OuterVolumeSpecName: "dshm") pod "f53d405d-1548-4f19-9dae-4f7abeab0c7b" (UID: "f53d405d-1548-4f19-9dae-4f7abeab0c7b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:24:55.838799 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.838756 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f53d405d-1548-4f19-9dae-4f7abeab0c7b" (UID: "f53d405d-1548-4f19-9dae-4f7abeab0c7b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:24:55.838799 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.838778 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f53d405d-1548-4f19-9dae-4f7abeab0c7b-kube-api-access-f9gnc" (OuterVolumeSpecName: "kube-api-access-f9gnc") pod "f53d405d-1548-4f19-9dae-4f7abeab0c7b" (UID: "f53d405d-1548-4f19-9dae-4f7abeab0c7b"). InnerVolumeSpecName "kube-api-access-f9gnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:24:55.937467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.937430 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f53d405d-1548-4f19-9dae-4f7abeab0c7b-tls-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:24:55.937467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.937462 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-home\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:24:55.937467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.937470 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-dshm\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:24:55.937467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.937479 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9gnc\" (UniqueName: \"kubernetes.io/projected/f53d405d-1548-4f19-9dae-4f7abeab0c7b-kube-api-access-f9gnc\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:24:55.937757 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:55.937488 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f53d405d-1548-4f19-9dae-4f7abeab0c7b-model-cache\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:24:56.550624 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.550564 2580 generic.go:356] "Generic (PLEG): container finished" podID="f53d405d-1548-4f19-9dae-4f7abeab0c7b" containerID="28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24" exitCode=0 Jul 18 02:24:56.551119 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.550635 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" Jul 18 02:24:56.551119 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.550643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" event={"ID":"f53d405d-1548-4f19-9dae-4f7abeab0c7b","Type":"ContainerDied","Data":"28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24"} Jul 18 02:24:56.551119 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.550683 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v" event={"ID":"f53d405d-1548-4f19-9dae-4f7abeab0c7b","Type":"ContainerDied","Data":"540ff7ae07f6531385606671b2ae40cb3f594f83b6c35264aff38ba58dcd877c"} Jul 18 02:24:56.551119 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.550700 2580 scope.go:117] "RemoveContainer" containerID="28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24" Jul 18 02:24:56.560114 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.560095 2580 scope.go:117] "RemoveContainer" containerID="28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24" Jul 18 02:24:56.560371 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:24:56.560351 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24\": container with ID starting with 28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24 not found: ID does not exist" containerID="28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24" Jul 18 02:24:56.560438 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.560378 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24"} err="failed to get container status \"28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24\": rpc error: code = NotFound desc = could not find container \"28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24\": container with ID starting with 28b69a665fa4d79afc49cfb24c10200ab0b6c4f84eb50a7f08fa83e4b31b8f24 not found: ID does not exist" Jul 18 02:24:56.572137 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.572111 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v"] Jul 18 02:24:56.575661 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.575634 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f21b76/test-model-f21b76-kserve-6b7795fcbf-hdr9v"] Jul 18 02:24:56.897611 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:24:56.897508 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f53d405d-1548-4f19-9dae-4f7abeab0c7b" path="/var/lib/kubelet/pods/f53d405d-1548-4f19-9dae-4f7abeab0c7b/volumes" Jul 18 02:25:16.002633 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:16.002599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:25:23.873602 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:23.873534 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x"] Jul 18 02:25:23.874084 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:23.873820 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" containerID="cri-o://85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a" gracePeriod=30 Jul 18 02:25:27.686229 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:27.686192 2580 patch_prober.go:28] interesting pod/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 02:25:27.686734 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:27.686261 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 02:25:29.128480 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.128446 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:25:29.236763 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236728 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-credential-socket\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.236937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236790 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc24e0a-00b7-42d3-9532-49b43095b183-istiod-ca-cert\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.236937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236808 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-socket\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.236937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236834 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89hb2\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-kube-api-access-89hb2\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.236937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236881 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-certs\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.236937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236918 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-envoy\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.236937 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236934 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-data\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.237232 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236967 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-podinfo\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.237232 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.236994 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-token\") pod \"4dc24e0a-00b7-42d3-9532-49b43095b183\" (UID: \"4dc24e0a-00b7-42d3-9532-49b43095b183\") " Jul 18 02:25:29.237232 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237105 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:29.237232 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237149 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:29.237232 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237175 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dc24e0a-00b7-42d3-9532-49b43095b183-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:25:29.237492 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237303 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-credential-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.237492 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237323 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dc24e0a-00b7-42d3-9532-49b43095b183-istiod-ca-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.237492 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237338 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.237492 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237409 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:29.237985 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.237954 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-data" (OuterVolumeSpecName: "istio-data") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:29.239301 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.239269 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-kube-api-access-89hb2" (OuterVolumeSpecName: "kube-api-access-89hb2") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "kube-api-access-89hb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:29.239301 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.239274 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:29.239454 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.239330 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:25:29.239454 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.239367 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-token" (OuterVolumeSpecName: "istio-token") pod "4dc24e0a-00b7-42d3-9532-49b43095b183" (UID: "4dc24e0a-00b7-42d3-9532-49b43095b183"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:29.337832 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.337797 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-workload-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.337832 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.337826 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-envoy\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.337832 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.337835 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-data\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.337832 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.337842 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-podinfo\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.338106 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.337850 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-istio-token\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.338106 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.337858 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89hb2\" (UniqueName: \"kubernetes.io/projected/4dc24e0a-00b7-42d3-9532-49b43095b183-kube-api-access-89hb2\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:25:29.721020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.720923 2580 generic.go:356] "Generic (PLEG): container finished" podID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerID="85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a" exitCode=0 Jul 18 02:25:29.721020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.720978 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" Jul 18 02:25:29.721020 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.721009 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" event={"ID":"4dc24e0a-00b7-42d3-9532-49b43095b183","Type":"ContainerDied","Data":"85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a"} Jul 18 02:25:29.721274 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.721066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x" event={"ID":"4dc24e0a-00b7-42d3-9532-49b43095b183","Type":"ContainerDied","Data":"d7b5b8e1acbd995992c16889dde9bb2197fffc52b49e072815839af04478d725"} Jul 18 02:25:29.721274 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.721090 2580 scope.go:117] "RemoveContainer" containerID="85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a" Jul 18 02:25:29.732952 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.732933 2580 scope.go:117] "RemoveContainer" containerID="463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548" Jul 18 02:25:29.741142 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.741120 2580 scope.go:117] "RemoveContainer" containerID="85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a" Jul 18 02:25:29.741458 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:25:29.741437 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a\": container with ID starting with 85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a not found: ID does not exist" containerID="85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a" Jul 18 02:25:29.741517 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.741469 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a"} err="failed to get container status \"85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a\": rpc error: code = NotFound desc = could not find container \"85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a\": container with ID starting with 85be1d99525ec04eb4dbccd2c5ec481efe349a8f5402edcbf1781d7150c4e93a not found: ID does not exist" Jul 18 02:25:29.741517 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.741488 2580 scope.go:117] "RemoveContainer" containerID="463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548" Jul 18 02:25:29.741772 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:25:29.741754 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548\": container with ID starting with 463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548 not found: ID does not exist" containerID="463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548" Jul 18 02:25:29.741823 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.741780 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548"} err="failed to get container status \"463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548\": rpc error: code = NotFound desc = could not find container \"463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548\": container with ID starting with 463cf8a1886c035a63ddd134b32aa5d266ebecd8ce6ee981d0eb3f79a9a4b548 not found: ID does not exist" Jul 18 02:25:29.747173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.747150 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x"] Jul 18 02:25:29.751646 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:29.751622 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f21b76-openshift-default-794695f5c5-gq92x"] Jul 18 02:25:30.897775 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:30.897743 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" path="/var/lib/kubelet/pods/4dc24e0a-00b7-42d3-9532-49b43095b183/volumes" Jul 18 02:25:33.109953 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.109918 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz"] Jul 18 02:25:33.110395 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.110381 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" Jul 18 02:25:33.110444 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.110399 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" Jul 18 02:25:33.110444 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.110420 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f53d405d-1548-4f19-9dae-4f7abeab0c7b" containerName="main" Jul 18 02:25:33.110444 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.110429 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f53d405d-1548-4f19-9dae-4f7abeab0c7b" containerName="main" Jul 18 02:25:33.110547 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.110516 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" Jul 18 02:25:33.110547 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.110527 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f53d405d-1548-4f19-9dae-4f7abeab0c7b" containerName="main" Jul 18 02:25:33.120691 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.120660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.123174 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.123147 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0e9c76-openshift-default-dockercfg-ntmqw\"" Jul 18 02:25:33.127625 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.127596 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz"] Jul 18 02:25:33.270100 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5k4l\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-kube-api-access-k5k4l\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270100 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-token\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270363 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istiod-ca-cert\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270363 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-data\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270363 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-credential-socket\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270363 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-podinfo\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270363 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-certs\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270552 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-envoy\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.270552 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.270446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-socket\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-podinfo\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-certs\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-envoy\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-socket\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5k4l\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-kube-api-access-k5k4l\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-token\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371493 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istiod-ca-cert\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-data\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.371640 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371568 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-credential-socket\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.372003 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371764 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-certs\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.372003 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-socket\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.372003 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.371951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-credential-socket\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.372164 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.372039 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-data\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.372336 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.372314 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istiod-ca-cert\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.373739 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.373717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-envoy\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.373961 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.373943 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-podinfo\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.380033 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.380004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-token\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.380416 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.380397 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5k4l\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-kube-api-access-k5k4l\") pod \"e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.434474 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.434436 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:33.570686 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.570656 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz"] Jul 18 02:25:33.572460 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:25:33.572428 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf23058_dbbe_4d92_aabc_0e2837d43bd3.slice/crio-35ef4684a74bcbf4ceb728680a7d2b414821030ab4bbb46092a1f9564c4ced7b WatchSource:0}: Error finding container 35ef4684a74bcbf4ceb728680a7d2b414821030ab4bbb46092a1f9564c4ced7b: Status 404 returned error can't find the container with id 35ef4684a74bcbf4ceb728680a7d2b414821030ab4bbb46092a1f9564c4ced7b Jul 18 02:25:33.574651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.574613 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:25:33.574756 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.574686 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:25:33.574756 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.574724 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:25:33.745474 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.745435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" event={"ID":"acf23058-dbbe-4d92-aabc-0e2837d43bd3","Type":"ContainerStarted","Data":"255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67"} Jul 18 02:25:33.745666 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.745497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" event={"ID":"acf23058-dbbe-4d92-aabc-0e2837d43bd3","Type":"ContainerStarted","Data":"35ef4684a74bcbf4ceb728680a7d2b414821030ab4bbb46092a1f9564c4ced7b"} Jul 18 02:25:33.763886 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:33.763823 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podStartSLOduration=0.76380209 podStartE2EDuration="763.80209ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:33.763456251 +0000 UTC m=+2239.447328990" watchObservedRunningTime="2026-07-18 02:25:33.76380209 +0000 UTC m=+2239.447674830" Jul 18 02:25:34.435620 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:34.435543 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:34.440464 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:34.440433 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:34.750931 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:34.750897 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:34.751971 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:34.751953 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:35.506476 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:35.506442 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:25:51.403856 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:51.403817 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:25:52.841856 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:52.841822 2580 generic.go:356] "Generic (PLEG): container finished" podID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerID="255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67" exitCode=0 Jul 18 02:25:52.842277 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:52.841881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" event={"ID":"acf23058-dbbe-4d92-aabc-0e2837d43bd3","Type":"ContainerDied","Data":"255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67"} Jul 18 02:25:52.842277 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:52.842251 2580 scope.go:117] "RemoveContainer" containerID="255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67" Jul 18 02:25:52.843225 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:52.843194 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:25:52.843291 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:52.843266 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:25:52.843328 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:52.843298 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:25:53.855036 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:53.854989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" event={"ID":"acf23058-dbbe-4d92-aabc-0e2837d43bd3","Type":"ContainerStarted","Data":"3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2"} Jul 18 02:25:54.435009 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:54.434966 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:54.435217 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:54.435193 2580 patch_prober.go:28] interesting pod/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:25:54.435279 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:54.435258 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:25:55.435791 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:55.435753 2580 patch_prober.go:28] interesting pod/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:25:55.436181 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:55.435808 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:25:56.435777 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:56.435738 2580 patch_prober.go:28] interesting pod/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:25:56.435975 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:56.435799 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:25:57.434970 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:57.434929 2580 patch_prober.go:28] interesting pod/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Jul 18 02:25:57.435157 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:57.434999 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Jul 18 02:25:58.438678 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:58.438644 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:58.439199 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:58.438902 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:25:58.439510 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:25:58.439493 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:26:06.601538 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:06.601500 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:26:34.900487 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:34.900447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:26:41.603634 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:41.603598 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz"] Jul 18 02:26:41.604129 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:41.603920 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" containerID="cri-o://3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2" gracePeriod=30 Jul 18 02:26:43.441070 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:43.441008 2580 patch_prober.go:28] interesting pod/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 02:26:43.441524 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:43.441094 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 02:26:46.858484 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:46.858456 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:26:47.030243 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030204 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5k4l\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-kube-api-access-k5k4l\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030415 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030270 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-credential-socket\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030415 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030344 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-data\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030415 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030374 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istiod-ca-cert\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030415 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030398 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-podinfo\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030659 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030426 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-token\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030659 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030448 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-socket\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030659 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030490 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-certs\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030659 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030560 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-envoy\") pod \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\" (UID: \"acf23058-dbbe-4d92-aabc-0e2837d43bd3\") " Jul 18 02:26:47.030870 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030748 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:47.030870 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030826 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:26:47.030977 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030925 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:47.031033 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.030978 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:47.031033 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.031002 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.031033 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.031025 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istiod-ca-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.031160 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.031039 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-workload-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.031310 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.031291 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-data" (OuterVolumeSpecName: "istio-data") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:47.033164 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.033127 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:26:47.033164 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.033128 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-token" (OuterVolumeSpecName: "istio-token") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:47.033312 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.033173 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:47.033312 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.033179 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-kube-api-access-k5k4l" (OuterVolumeSpecName: "kube-api-access-k5k4l") pod "acf23058-dbbe-4d92-aabc-0e2837d43bd3" (UID: "acf23058-dbbe-4d92-aabc-0e2837d43bd3"). InnerVolumeSpecName "kube-api-access-k5k4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:47.126555 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.126512 2580 generic.go:356] "Generic (PLEG): container finished" podID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerID="3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2" exitCode=0 Jul 18 02:26:47.126555 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.126559 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" Jul 18 02:26:47.126773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.126568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" event={"ID":"acf23058-dbbe-4d92-aabc-0e2837d43bd3","Type":"ContainerDied","Data":"3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2"} Jul 18 02:26:47.126773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.126644 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz" event={"ID":"acf23058-dbbe-4d92-aabc-0e2837d43bd3","Type":"ContainerDied","Data":"35ef4684a74bcbf4ceb728680a7d2b414821030ab4bbb46092a1f9564c4ced7b"} Jul 18 02:26:47.126773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.126672 2580 scope.go:117] "RemoveContainer" containerID="3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2" Jul 18 02:26:47.131895 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.131874 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-envoy\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.131895 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.131894 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5k4l\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-kube-api-access-k5k4l\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.132036 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.131904 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-credential-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.132036 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.131915 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-data\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.132036 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.131923 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-podinfo\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.132036 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.131931 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/acf23058-dbbe-4d92-aabc-0e2837d43bd3-istio-token\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:26:47.136239 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.136209 2580 scope.go:117] "RemoveContainer" containerID="255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67" Jul 18 02:26:47.148068 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.148051 2580 scope.go:117] "RemoveContainer" containerID="3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2" Jul 18 02:26:47.148347 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:26:47.148327 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2\": container with ID starting with 3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2 not found: ID does not exist" containerID="3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2" Jul 18 02:26:47.148413 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.148354 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2"} err="failed to get container status \"3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2\": rpc error: code = NotFound desc = could not find container \"3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2\": container with ID starting with 3b8a3beb0bd2924df687caa9a9c1ec7156ff3bc1860718194d28e59bb82d11f2 not found: ID does not exist" Jul 18 02:26:47.148413 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.148371 2580 scope.go:117] "RemoveContainer" containerID="255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67" Jul 18 02:26:47.148630 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:26:47.148611 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67\": container with ID starting with 255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67 not found: ID does not exist" containerID="255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67" Jul 18 02:26:47.148684 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.148636 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67"} err="failed to get container status \"255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67\": rpc error: code = NotFound desc = could not find container \"255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67\": container with ID starting with 255f882d20f38563604899754b4e25621c407406df290d5679f2c6ea242edd67 not found: ID does not exist" Jul 18 02:26:47.155241 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.155092 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz"] Jul 18 02:26:47.157202 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:47.157179 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0e9c76-openshift-default-75896b8565-vjbrz"] Jul 18 02:26:48.898642 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:48.898612 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" path="/var/lib/kubelet/pods/acf23058-dbbe-4d92-aabc-0e2837d43bd3/volumes" Jul 18 02:26:50.418915 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.418877 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk"] Jul 18 02:26:50.419345 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419329 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" Jul 18 02:26:50.419345 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419343 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" Jul 18 02:26:50.419472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419369 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" Jul 18 02:26:50.419472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419378 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" Jul 18 02:26:50.419472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419396 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" Jul 18 02:26:50.419472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419404 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" Jul 18 02:26:50.419714 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419474 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dc24e0a-00b7-42d3-9532-49b43095b183" containerName="istio-proxy" Jul 18 02:26:50.419714 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.419485 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" Jul 18 02:26:50.421780 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.421756 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.423958 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.423939 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-f0c43e-openshift-default-dockercfg-dhvw2\"" Jul 18 02:26:50.435529 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.435509 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk"] Jul 18 02:26:50.454291 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086a5786-b610-4781-9eb4-583dc599fe7e-istiod-ca-cert\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454291 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54nvk\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-kube-api-access-54nvk\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454504 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-envoy\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454504 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-credential-socket\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454624 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-certs\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454624 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454566 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-socket\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086a5786-b610-4781-9eb4-583dc599fe7e-istio-podinfo\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454673 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-istio-token\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.454773 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.454693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-data\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.555913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.555880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-envoy\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556110 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.555921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-credential-socket\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556110 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.555953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-certs\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556110 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-socket\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556282 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086a5786-b610-4781-9eb4-583dc599fe7e-istio-podinfo\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556282 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-istio-token\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556282 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-data\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556282 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086a5786-b610-4781-9eb4-583dc599fe7e-istiod-ca-cert\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556282 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54nvk\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-kube-api-access-54nvk\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556518 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556327 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-certs\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556518 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556404 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-credential-socket\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556657 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556634 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-data\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.556921 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.556899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-socket\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.557168 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.557150 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086a5786-b610-4781-9eb4-583dc599fe7e-istiod-ca-cert\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.558288 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.558265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-envoy\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.558651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.558632 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086a5786-b610-4781-9eb4-583dc599fe7e-istio-podinfo\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.563689 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.563668 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-istio-token\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.563836 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.563817 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54nvk\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-kube-api-access-54nvk\") pod \"e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.734664 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.734626 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:50.877232 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.877194 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk"] Jul 18 02:26:50.880419 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:26:50.880388 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod086a5786_b610_4781_9eb4_583dc599fe7e.slice/crio-d223627fa080eb7a7823099cd6ae9a847b024f62176cbb94b50680d72a2126ba WatchSource:0}: Error finding container d223627fa080eb7a7823099cd6ae9a847b024f62176cbb94b50680d72a2126ba: Status 404 returned error can't find the container with id d223627fa080eb7a7823099cd6ae9a847b024f62176cbb94b50680d72a2126ba Jul 18 02:26:50.882471 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.882434 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:26:50.882535 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.882508 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:26:50.882573 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:50.882543 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:26:51.150098 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:51.150006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" event={"ID":"086a5786-b610-4781-9eb4-583dc599fe7e","Type":"ContainerStarted","Data":"a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909"} Jul 18 02:26:51.150098 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:51.150063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" event={"ID":"086a5786-b610-4781-9eb4-583dc599fe7e","Type":"ContainerStarted","Data":"d223627fa080eb7a7823099cd6ae9a847b024f62176cbb94b50680d72a2126ba"} Jul 18 02:26:51.168908 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:51.168853 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podStartSLOduration=1.16883807 podStartE2EDuration="1.16883807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:51.16661198 +0000 UTC m=+2316.850484721" watchObservedRunningTime="2026-07-18 02:26:51.16883807 +0000 UTC m=+2316.852710808" Jul 18 02:26:51.402104 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:51.402002 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:26:51.735425 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:51.735391 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:51.740483 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:51.740454 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:52.155769 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:52.155680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:26:52.156693 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:26:52.156669 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:27:07.100941 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.100895 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:27:07.156980 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.156938 2580 patch_prober.go:28] interesting pod/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:27:07.157183 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.157013 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:27:07.233516 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.233480 2580 generic.go:356] "Generic (PLEG): container finished" podID="086a5786-b610-4781-9eb4-583dc599fe7e" containerID="a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909" exitCode=0 Jul 18 02:27:07.233708 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.233531 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" event={"ID":"086a5786-b610-4781-9eb4-583dc599fe7e","Type":"ContainerDied","Data":"a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909"} Jul 18 02:27:07.234009 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.233989 2580 scope.go:117] "RemoveContainer" containerID="a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909" Jul 18 02:27:07.235014 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.234986 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:27:07.235068 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.235058 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:27:07.235114 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:07.235091 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 18 02:27:08.240260 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:08.240221 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" event={"ID":"086a5786-b610-4781-9eb4-583dc599fe7e","Type":"ContainerStarted","Data":"a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139"} Jul 18 02:27:08.735100 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:08.735061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:27:08.735324 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:08.735297 2580 patch_prober.go:28] interesting pod/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:27:08.735385 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:08.735354 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:27:09.735705 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:09.735668 2580 patch_prober.go:28] interesting pod/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:27:09.736118 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:09.735730 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:27:10.736095 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:10.736052 2580 patch_prober.go:28] interesting pod/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:27:10.736518 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:10.736120 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:27:11.735686 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:11.735644 2580 patch_prober.go:28] interesting pod/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 18 02:27:11.735873 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:11.735703 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 18 02:27:12.739234 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:12.739202 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:27:12.739704 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:12.739550 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:27:12.740387 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:12.740369 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:27:23.696635 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:23.696595 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:27:52.097936 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:27:52.097900 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:28:14.975129 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:28:14.975098 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:28:14.979729 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:28:14.979703 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:28:16.901381 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:28:16.901340 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:28:34.903097 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:28:34.903061 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:28:51.694455 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:28:51.694415 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:29:08.198059 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:29:08.198018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:29:26.596620 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:29:26.596564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:29:40.501769 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:29:40.501726 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:29:59.597282 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:29:59.597239 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:30:05.197602 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.197522 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn"] Jul 18 02:30:05.198317 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.198291 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="acf23058-dbbe-4d92-aabc-0e2837d43bd3" containerName="istio-proxy" Jul 18 02:30:05.202014 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.201982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.205151 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.205104 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"default-dockercfg-vh9tm\"" Jul 18 02:30:05.205404 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.205230 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"openshift-service-ca.crt\"" Jul 18 02:30:05.205404 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.205396 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"sub-test-model-d05e31-kserve-self-signed-certs\"" Jul 18 02:30:05.205495 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.205108 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-d05e31\"/\"kube-root-ca.crt\"" Jul 18 02:30:05.213250 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.213223 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn"] Jul 18 02:30:05.323532 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.323481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tls-certs\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.323748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.323543 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-home\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.323748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.323627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-dshm\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.323748 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.323667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tmp-dir\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.323861 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.323770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-model-cache\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.323861 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.323806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79g2l\" (UniqueName: \"kubernetes.io/projected/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-kube-api-access-79g2l\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425256 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-dshm\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425256 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tmp-dir\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425495 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-model-cache\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425495 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79g2l\" (UniqueName: \"kubernetes.io/projected/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-kube-api-access-79g2l\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425495 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tls-certs\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425495 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-home\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425761 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tmp-dir\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-model-cache\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.425839 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.425828 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-home\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.427633 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.427605 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-dshm\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.427912 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.427894 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tls-certs\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.433504 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.433475 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79g2l\" (UniqueName: \"kubernetes.io/projected/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-kube-api-access-79g2l\") pod \"sub-test-model-d05e31-kserve-8445f84d59-7kjbn\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.515770 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.515727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:05.660693 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:05.660663 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn"] Jul 18 02:30:05.663189 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:30:05.663149 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bce31d8_8191_42ab_8c7d_bb3836f0df8c.slice/crio-c2d9be425afd8eac67ef86ab2cf0a7ac18c7e6a7aefecf5840d6013b1d62771e WatchSource:0}: Error finding container c2d9be425afd8eac67ef86ab2cf0a7ac18c7e6a7aefecf5840d6013b1d62771e: Status 404 returned error can't find the container with id c2d9be425afd8eac67ef86ab2cf0a7ac18c7e6a7aefecf5840d6013b1d62771e Jul 18 02:30:06.152872 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:06.152831 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" event={"ID":"6bce31d8-8191-42ab-8c7d-bb3836f0df8c","Type":"ContainerStarted","Data":"d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30"} Jul 18 02:30:06.152872 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:06.152886 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" event={"ID":"6bce31d8-8191-42ab-8c7d-bb3836f0df8c","Type":"ContainerStarted","Data":"c2d9be425afd8eac67ef86ab2cf0a7ac18c7e6a7aefecf5840d6013b1d62771e"} Jul 18 02:30:06.171506 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:06.171444 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" podStartSLOduration=1.171425916 podStartE2EDuration="1.171425916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:30:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:30:06.169055172 +0000 UTC m=+2511.852927912" watchObservedRunningTime="2026-07-18 02:30:06.171425916 +0000 UTC m=+2511.855298756" Jul 18 02:30:13.303688 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:13.303594 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:30:15.515982 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:15.515934 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:15.516404 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:15.516020 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:15.528612 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:15.528565 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:16.214975 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:16.214943 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:30:34.995570 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:34.995533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:30:49.502223 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:30:49.502174 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:31:08.106780 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:08.106739 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:31:13.509411 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:13.509373 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn"] Jul 18 02:31:25.298286 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:25.298247 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:31:28.530342 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.530301 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" podUID="6bce31d8-8191-42ab-8c7d-bb3836f0df8c" containerName="main" containerID="cri-o://d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30" gracePeriod=45 Jul 18 02:31:28.790339 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.790261 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:31:28.869472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869427 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-model-cache\") pod \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " Jul 18 02:31:28.869689 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869501 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-dshm\") pod \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " Jul 18 02:31:28.869689 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869524 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79g2l\" (UniqueName: \"kubernetes.io/projected/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-kube-api-access-79g2l\") pod \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " Jul 18 02:31:28.869689 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869561 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tls-certs\") pod \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " Jul 18 02:31:28.869689 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869616 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tmp-dir\") pod \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " Jul 18 02:31:28.869689 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869665 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-home\") pod \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\" (UID: \"6bce31d8-8191-42ab-8c7d-bb3836f0df8c\") " Jul 18 02:31:28.869961 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869776 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-model-cache" (OuterVolumeSpecName: "model-cache") pod "6bce31d8-8191-42ab-8c7d-bb3836f0df8c" (UID: "6bce31d8-8191-42ab-8c7d-bb3836f0df8c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:28.869961 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869946 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6bce31d8-8191-42ab-8c7d-bb3836f0df8c" (UID: "6bce31d8-8191-42ab-8c7d-bb3836f0df8c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:28.870055 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869968 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-model-cache\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:31:28.870055 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.869985 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-home" (OuterVolumeSpecName: "home") pod "6bce31d8-8191-42ab-8c7d-bb3836f0df8c" (UID: "6bce31d8-8191-42ab-8c7d-bb3836f0df8c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:28.871916 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.871888 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-dshm" (OuterVolumeSpecName: "dshm") pod "6bce31d8-8191-42ab-8c7d-bb3836f0df8c" (UID: "6bce31d8-8191-42ab-8c7d-bb3836f0df8c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:31:28.871992 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.871936 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-kube-api-access-79g2l" (OuterVolumeSpecName: "kube-api-access-79g2l") pod "6bce31d8-8191-42ab-8c7d-bb3836f0df8c" (UID: "6bce31d8-8191-42ab-8c7d-bb3836f0df8c"). InnerVolumeSpecName "kube-api-access-79g2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:31:28.872030 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.872003 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6bce31d8-8191-42ab-8c7d-bb3836f0df8c" (UID: "6bce31d8-8191-42ab-8c7d-bb3836f0df8c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:31:28.970407 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.970361 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-dshm\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:31:28.970407 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.970398 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79g2l\" (UniqueName: \"kubernetes.io/projected/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-kube-api-access-79g2l\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:31:28.970407 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.970411 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tls-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:31:28.970407 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.970420 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-tmp-dir\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:31:28.970730 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:28.970427 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bce31d8-8191-42ab-8c7d-bb3836f0df8c-home\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:31:29.584125 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.584090 2580 generic.go:356] "Generic (PLEG): container finished" podID="6bce31d8-8191-42ab-8c7d-bb3836f0df8c" containerID="d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30" exitCode=0 Jul 18 02:31:29.584592 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.584133 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" Jul 18 02:31:29.584592 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.584151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" event={"ID":"6bce31d8-8191-42ab-8c7d-bb3836f0df8c","Type":"ContainerDied","Data":"d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30"} Jul 18 02:31:29.584592 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.584197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn" event={"ID":"6bce31d8-8191-42ab-8c7d-bb3836f0df8c","Type":"ContainerDied","Data":"c2d9be425afd8eac67ef86ab2cf0a7ac18c7e6a7aefecf5840d6013b1d62771e"} Jul 18 02:31:29.584592 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.584214 2580 scope.go:117] "RemoveContainer" containerID="d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30" Jul 18 02:31:29.593490 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.593466 2580 scope.go:117] "RemoveContainer" containerID="d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30" Jul 18 02:31:29.593820 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:31:29.593796 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30\": container with ID starting with d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30 not found: ID does not exist" containerID="d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30" Jul 18 02:31:29.593908 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.593829 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30"} err="failed to get container status \"d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30\": rpc error: code = NotFound desc = could not find container \"d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30\": container with ID starting with d88c1401a738cebef1aac10a374c6d0a24749ad85ea2f92b33238fab81fc4e30 not found: ID does not exist" Jul 18 02:31:29.601438 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.601402 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn"] Jul 18 02:31:29.603474 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:29.603447 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-d05e31/sub-test-model-d05e31-kserve-8445f84d59-7kjbn"] Jul 18 02:31:30.897250 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:30.897211 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bce31d8-8191-42ab-8c7d-bb3836f0df8c" path="/var/lib/kubelet/pods/6bce31d8-8191-42ab-8c7d-bb3836f0df8c/volumes" Jul 18 02:31:36.298485 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:31:36.298442 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:32:24.802535 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:32:24.802489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:32:38.904576 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:32:38.904542 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:32:54.000257 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:32:54.000216 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:33:03.052913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.052868 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7434-140083572527664"] Jul 18 02:33:03.053410 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.053351 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bce31d8-8191-42ab-8c7d-bb3836f0df8c" containerName="main" Jul 18 02:33:03.053410 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.053370 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bce31d8-8191-42ab-8c7d-bb3836f0df8c" containerName="main" Jul 18 02:33:03.053533 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.053455 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bce31d8-8191-42ab-8c7d-bb3836f0df8c" containerName="main" Jul 18 02:33:03.056438 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.056415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:03.059097 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.059069 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tkmd9\"" Jul 18 02:33:03.064642 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.064615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572527664"] Jul 18 02:33:03.195601 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.195545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x68j4\" (UniqueName: \"kubernetes.io/projected/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6-kube-api-access-x68j4\") pod \"test-curl-7434-140083572527664\" (UID: \"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6\") " pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:03.296961 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.296926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x68j4\" (UniqueName: \"kubernetes.io/projected/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6-kube-api-access-x68j4\") pod \"test-curl-7434-140083572527664\" (UID: \"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6\") " pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:03.311408 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.311332 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x68j4\" (UniqueName: \"kubernetes.io/projected/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6-kube-api-access-x68j4\") pod \"test-curl-7434-140083572527664\" (UID: \"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6\") " pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:03.367108 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.367071 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:03.502914 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.502881 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572527664"] Jul 18 02:33:03.504418 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:33:03.504393 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5dc1d1d1_298b_4793_b8fa_7cf7161a3bb6.slice/crio-90565b638c074ed5c22fc44e554a45e806d0d35f733fcfd1af5c183f53ed8ea5 WatchSource:0}: Error finding container 90565b638c074ed5c22fc44e554a45e806d0d35f733fcfd1af5c183f53ed8ea5: Status 404 returned error can't find the container with id 90565b638c074ed5c22fc44e554a45e806d0d35f733fcfd1af5c183f53ed8ea5 Jul 18 02:33:03.505810 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:03.505793 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:33:04.058842 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:04.058804 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572527664" event={"ID":"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6","Type":"ContainerStarted","Data":"90565b638c074ed5c22fc44e554a45e806d0d35f733fcfd1af5c183f53ed8ea5"} Jul 18 02:33:05.065619 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:05.065563 2580 generic.go:356] "Generic (PLEG): container finished" podID="5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6" containerID="68cacae5322f3fbe260941be159d924285c48c338f3f2f174f3709557c1f51e6" exitCode=0 Jul 18 02:33:05.066008 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:05.065651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572527664" event={"ID":"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6","Type":"ContainerDied","Data":"68cacae5322f3fbe260941be159d924285c48c338f3f2f174f3709557c1f51e6"} Jul 18 02:33:06.210668 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.210634 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:06.245779 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.245748 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7434-140083572527664_5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6/test-curl-7434-140083572527664/0.log" Jul 18 02:33:06.268280 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.268240 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7434-140083572527664"] Jul 18 02:33:06.271031 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.271001 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7434-140083572527664"] Jul 18 02:33:06.323036 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.323002 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x68j4\" (UniqueName: \"kubernetes.io/projected/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6-kube-api-access-x68j4\") pod \"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6\" (UID: \"5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6\") " Jul 18 02:33:06.325244 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.325214 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6-kube-api-access-x68j4" (OuterVolumeSpecName: "kube-api-access-x68j4") pod "5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6" (UID: "5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6"). InnerVolumeSpecName "kube-api-access-x68j4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:06.424247 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.424203 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x68j4\" (UniqueName: \"kubernetes.io/projected/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6-kube-api-access-x68j4\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:33:06.897595 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:06.897546 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6" path="/var/lib/kubelet/pods/5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6/volumes" Jul 18 02:33:07.081858 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:07.081827 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572527664" Jul 18 02:33:07.082043 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:07.081890 2580 scope.go:117] "RemoveContainer" containerID="68cacae5322f3fbe260941be159d924285c48c338f3f2f174f3709557c1f51e6" Jul 18 02:33:08.202260 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:08.202218 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:33:09.843077 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.843037 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:09.843501 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.843375 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6" containerName="test-curl-7434-140083572527664" Jul 18 02:33:09.843501 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.843387 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6" containerName="test-curl-7434-140083572527664" Jul 18 02:33:09.843501 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.843440 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc1d1d1-298b-4793-b8fa-7cf7161a3bb6" containerName="test-curl-7434-140083572527664" Jul 18 02:33:09.847960 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.847935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:09.850480 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.850450 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tkmd9\"" Jul 18 02:33:09.852664 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.852637 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:09.948653 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:09.948616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqblk\" (UniqueName: \"kubernetes.io/projected/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f-kube-api-access-pqblk\") pod \"test-curl-7434-140083572478864\" (UID: \"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f\") " pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:10.049761 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:10.049719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqblk\" (UniqueName: \"kubernetes.io/projected/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f-kube-api-access-pqblk\") pod \"test-curl-7434-140083572478864\" (UID: \"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f\") " pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:10.057969 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:10.057943 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqblk\" (UniqueName: \"kubernetes.io/projected/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f-kube-api-access-pqblk\") pod \"test-curl-7434-140083572478864\" (UID: \"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f\") " pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:10.159525 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:10.159430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:10.294795 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:10.294769 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:10.296508 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:33:10.296482 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c6dc3a3_3f29_4a44_ac7e_ff4f5d75f84f.slice/crio-b12a8378da057c15b3abbc3b195c316f77b3f8b78c7f29060f8d04bfcc273abd WatchSource:0}: Error finding container b12a8378da057c15b3abbc3b195c316f77b3f8b78c7f29060f8d04bfcc273abd: Status 404 returned error can't find the container with id b12a8378da057c15b3abbc3b195c316f77b3f8b78c7f29060f8d04bfcc273abd Jul 18 02:33:11.104995 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:11.104954 2580 generic.go:356] "Generic (PLEG): container finished" podID="9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f" containerID="35ee05ca8ad18c8ea0e14efaca1f47d7161907eb5cd353f518dbdb89dd037946" exitCode=0 Jul 18 02:33:11.105389 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:11.105041 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572478864" event={"ID":"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f","Type":"ContainerDied","Data":"35ee05ca8ad18c8ea0e14efaca1f47d7161907eb5cd353f518dbdb89dd037946"} Jul 18 02:33:11.105389 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:11.105081 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572478864" event={"ID":"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f","Type":"ContainerStarted","Data":"b12a8378da057c15b3abbc3b195c316f77b3f8b78c7f29060f8d04bfcc273abd"} Jul 18 02:33:12.251014 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.250985 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:12.267960 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.267928 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqblk\" (UniqueName: \"kubernetes.io/projected/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f-kube-api-access-pqblk\") pod \"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f\" (UID: \"9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f\") " Jul 18 02:33:12.274991 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.274947 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f-kube-api-access-pqblk" (OuterVolumeSpecName: "kube-api-access-pqblk") pod "9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f" (UID: "9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f"). InnerVolumeSpecName "kube-api-access-pqblk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:12.278725 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.278460 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7434-140083572478864_9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f/test-curl-7434-140083572478864/0.log" Jul 18 02:33:12.299048 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.299009 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:12.302502 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.302467 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:12.369421 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.369330 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqblk\" (UniqueName: \"kubernetes.io/projected/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f-kube-api-access-pqblk\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:33:12.418564 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.418525 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:12.419106 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.419087 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f" containerName="test-curl-7434-140083572478864" Jul 18 02:33:12.419169 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.419108 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f" containerName="test-curl-7434-140083572478864" Jul 18 02:33:12.419216 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.419187 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f" containerName="test-curl-7434-140083572478864" Jul 18 02:33:12.422365 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.422340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:12.427227 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.427201 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:12.469875 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.469838 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcqbk\" (UniqueName: \"kubernetes.io/projected/a3705d87-9b57-47bd-972d-3a97bc8bfbac-kube-api-access-jcqbk\") pod \"test-curl-7434-140083572478864\" (UID: \"a3705d87-9b57-47bd-972d-3a97bc8bfbac\") " pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:12.570606 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.570547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcqbk\" (UniqueName: \"kubernetes.io/projected/a3705d87-9b57-47bd-972d-3a97bc8bfbac-kube-api-access-jcqbk\") pod \"test-curl-7434-140083572478864\" (UID: \"a3705d87-9b57-47bd-972d-3a97bc8bfbac\") " pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:12.578221 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.578193 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcqbk\" (UniqueName: \"kubernetes.io/projected/a3705d87-9b57-47bd-972d-3a97bc8bfbac-kube-api-access-jcqbk\") pod \"test-curl-7434-140083572478864\" (UID: \"a3705d87-9b57-47bd-972d-3a97bc8bfbac\") " pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:12.733800 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.733766 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:12.867725 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.867696 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:12.869564 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:33:12.869539 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3705d87_9b57_47bd_972d_3a97bc8bfbac.slice/crio-21c60f1d550ca341af8885710243d9df573fc6961879274e9a869e574d00b417 WatchSource:0}: Error finding container 21c60f1d550ca341af8885710243d9df573fc6961879274e9a869e574d00b417: Status 404 returned error can't find the container with id 21c60f1d550ca341af8885710243d9df573fc6961879274e9a869e574d00b417 Jul 18 02:33:12.897565 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:12.897531 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f" path="/var/lib/kubelet/pods/9c6dc3a3-3f29-4a44-ac7e-ff4f5d75f84f/volumes" Jul 18 02:33:13.116470 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:13.116377 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:13.116470 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:13.116390 2580 scope.go:117] "RemoveContainer" containerID="35ee05ca8ad18c8ea0e14efaca1f47d7161907eb5cd353f518dbdb89dd037946" Jul 18 02:33:13.117794 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:13.117521 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572478864" event={"ID":"a3705d87-9b57-47bd-972d-3a97bc8bfbac","Type":"ContainerStarted","Data":"21c60f1d550ca341af8885710243d9df573fc6961879274e9a869e574d00b417"} Jul 18 02:33:14.124246 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:14.124213 2580 generic.go:356] "Generic (PLEG): container finished" podID="a3705d87-9b57-47bd-972d-3a97bc8bfbac" containerID="fa182f147b4a72efce964d214f0d2dfdc4c483d05652adf57a7e70a6843d7ef7" exitCode=0 Jul 18 02:33:14.124672 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:14.124305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572478864" event={"ID":"a3705d87-9b57-47bd-972d-3a97bc8bfbac","Type":"ContainerDied","Data":"fa182f147b4a72efce964d214f0d2dfdc4c483d05652adf57a7e70a6843d7ef7"} Jul 18 02:33:15.006913 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.006883 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:33:15.012634 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.012607 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-kcwzg_880e0b4b-9383-434f-8338-9834f1240186/ovn-acl-logging/0.log" Jul 18 02:33:15.278197 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.278165 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:15.294087 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.294060 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcqbk\" (UniqueName: \"kubernetes.io/projected/a3705d87-9b57-47bd-972d-3a97bc8bfbac-kube-api-access-jcqbk\") pod \"a3705d87-9b57-47bd-972d-3a97bc8bfbac\" (UID: \"a3705d87-9b57-47bd-972d-3a97bc8bfbac\") " Jul 18 02:33:15.296151 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.296123 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3705d87-9b57-47bd-972d-3a97bc8bfbac-kube-api-access-jcqbk" (OuterVolumeSpecName: "kube-api-access-jcqbk") pod "a3705d87-9b57-47bd-972d-3a97bc8bfbac" (UID: "a3705d87-9b57-47bd-972d-3a97bc8bfbac"). InnerVolumeSpecName "kube-api-access-jcqbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:15.297199 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.297175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7434-140083572478864_a3705d87-9b57-47bd-972d-3a97bc8bfbac/test-curl-7434-140083572478864/0.log" Jul 18 02:33:15.320901 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.320845 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:15.323059 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.323029 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7434-140083572478864"] Jul 18 02:33:15.395100 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.395055 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcqbk\" (UniqueName: \"kubernetes.io/projected/a3705d87-9b57-47bd-972d-3a97bc8bfbac-kube-api-access-jcqbk\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:33:15.437308 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.437269 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7434-140083572479872"] Jul 18 02:33:15.437662 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.437647 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3705d87-9b57-47bd-972d-3a97bc8bfbac" containerName="test-curl-7434-140083572478864" Jul 18 02:33:15.437712 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.437663 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3705d87-9b57-47bd-972d-3a97bc8bfbac" containerName="test-curl-7434-140083572478864" Jul 18 02:33:15.437747 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.437719 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3705d87-9b57-47bd-972d-3a97bc8bfbac" containerName="test-curl-7434-140083572478864" Jul 18 02:33:15.440686 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.440654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:15.447626 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.447564 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"] Jul 18 02:33:15.495517 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.495480 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwwzc\" (UniqueName: \"kubernetes.io/projected/15014020-95cb-4ce2-b69a-ef69df250096-kube-api-access-dwwzc\") pod \"test-curl-7434-140083572479872\" (UID: \"15014020-95cb-4ce2-b69a-ef69df250096\") " pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:15.596976 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.596873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwwzc\" (UniqueName: \"kubernetes.io/projected/15014020-95cb-4ce2-b69a-ef69df250096-kube-api-access-dwwzc\") pod \"test-curl-7434-140083572479872\" (UID: \"15014020-95cb-4ce2-b69a-ef69df250096\") " pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:15.605121 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.605086 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwwzc\" (UniqueName: \"kubernetes.io/projected/15014020-95cb-4ce2-b69a-ef69df250096-kube-api-access-dwwzc\") pod \"test-curl-7434-140083572479872\" (UID: \"15014020-95cb-4ce2-b69a-ef69df250096\") " pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:15.752134 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.752094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:15.881610 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:15.881561 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"] Jul 18 02:33:15.883051 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:33:15.883014 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15014020_95cb_4ce2_b69a_ef69df250096.slice/crio-bd8c66795c0665817f97c506c327b384a935c7ef7b6d4c6d849ab8a48ade4eb0 WatchSource:0}: Error finding container bd8c66795c0665817f97c506c327b384a935c7ef7b6d4c6d849ab8a48ade4eb0: Status 404 returned error can't find the container with id bd8c66795c0665817f97c506c327b384a935c7ef7b6d4c6d849ab8a48ade4eb0 Jul 18 02:33:16.135851 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:16.135762 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:16.135851 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:16.135771 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21c60f1d550ca341af8885710243d9df573fc6961879274e9a869e574d00b417" Jul 18 02:33:16.136912 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:16.136879 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572479872" event={"ID":"15014020-95cb-4ce2-b69a-ef69df250096","Type":"ContainerStarted","Data":"bd8c66795c0665817f97c506c327b384a935c7ef7b6d4c6d849ab8a48ade4eb0"} Jul 18 02:33:16.146436 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:33:16.146400 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7434-140083572478864\" is forbidden: User \"system:node:ip-10-0-137-56.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-56.ec2.internal' and this object" podUID="a3705d87-9b57-47bd-972d-3a97bc8bfbac" pod="opendatahub/test-curl-7434-140083572478864" Jul 18 02:33:16.897096 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:16.897060 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3705d87-9b57-47bd-972d-3a97bc8bfbac" path="/var/lib/kubelet/pods/a3705d87-9b57-47bd-972d-3a97bc8bfbac/volumes" Jul 18 02:33:17.142868 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:17.142836 2580 generic.go:356] "Generic (PLEG): container finished" podID="15014020-95cb-4ce2-b69a-ef69df250096" containerID="1e75256e92d945bff90ccf5791ebf476a993d19603027fd8626ee17ba8fc6030" exitCode=0 Jul 18 02:33:17.143076 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:17.142906 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572479872" event={"ID":"15014020-95cb-4ce2-b69a-ef69df250096","Type":"ContainerDied","Data":"1e75256e92d945bff90ccf5791ebf476a993d19603027fd8626ee17ba8fc6030"} Jul 18 02:33:18.288335 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.288308 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:18.307351 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.307325 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7434-140083572479872_15014020-95cb-4ce2-b69a-ef69df250096/test-curl-7434-140083572479872/0.log" Jul 18 02:33:18.320775 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.320744 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwwzc\" (UniqueName: \"kubernetes.io/projected/15014020-95cb-4ce2-b69a-ef69df250096-kube-api-access-dwwzc\") pod \"15014020-95cb-4ce2-b69a-ef69df250096\" (UID: \"15014020-95cb-4ce2-b69a-ef69df250096\") " Jul 18 02:33:18.322892 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.322860 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15014020-95cb-4ce2-b69a-ef69df250096-kube-api-access-dwwzc" (OuterVolumeSpecName: "kube-api-access-dwwzc") pod "15014020-95cb-4ce2-b69a-ef69df250096" (UID: "15014020-95cb-4ce2-b69a-ef69df250096"). InnerVolumeSpecName "kube-api-access-dwwzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:18.330709 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.330679 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"] Jul 18 02:33:18.332631 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.332606 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7434-140083572479872"] Jul 18 02:33:18.421635 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.421515 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwwzc\" (UniqueName: \"kubernetes.io/projected/15014020-95cb-4ce2-b69a-ef69df250096-kube-api-access-dwwzc\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:33:18.897779 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:18.897747 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15014020-95cb-4ce2-b69a-ef69df250096" path="/var/lib/kubelet/pods/15014020-95cb-4ce2-b69a-ef69df250096/volumes" Jul 18 02:33:19.153941 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:19.153854 2580 scope.go:117] "RemoveContainer" containerID="1e75256e92d945bff90ccf5791ebf476a993d19603027fd8626ee17ba8fc6030" Jul 18 02:33:19.154106 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:19.153860 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572479872" Jul 18 02:33:34.901911 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:34.901882 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:33:56.900449 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:33:56.900416 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clrdc"] Jul 18 02:34:05.320364 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:05.320329 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk"] Jul 18 02:34:05.320832 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:05.320527 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" containerID="cri-o://a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139" gracePeriod=30 Jul 18 02:34:10.589706 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.589623 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:34:10.665733 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.665692 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-data\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.665733 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.665733 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086a5786-b610-4781-9eb4-583dc599fe7e-istiod-ca-cert\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.665993 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.665780 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-socket\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.665993 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.665797 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086a5786-b610-4781-9eb4-583dc599fe7e-istio-podinfo\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.665993 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.665826 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-istio-token\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.665993 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.665885 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54nvk\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-kube-api-access-54nvk\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.666209 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666009 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-certs\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.666209 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666069 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:10.666209 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666074 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-envoy\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.666209 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666134 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-credential-socket\") pod \"086a5786-b610-4781-9eb4-583dc599fe7e\" (UID: \"086a5786-b610-4781-9eb4-583dc599fe7e\") " Jul 18 02:34:10.666405 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666308 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:10.666456 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666411 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-certs\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.666456 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666431 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-workload-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.666551 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666480 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:10.666631 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666552 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/086a5786-b610-4781-9eb4-583dc599fe7e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:34:10.666789 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.666764 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-data" (OuterVolumeSpecName: "istio-data") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:10.668212 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.668173 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-istio-token" (OuterVolumeSpecName: "istio-token") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:34:10.668212 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.668203 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-kube-api-access-54nvk" (OuterVolumeSpecName: "kube-api-access-54nvk") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "kube-api-access-54nvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:34:10.668476 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.668455 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:34:10.668476 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.668461 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/086a5786-b610-4781-9eb4-583dc599fe7e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "086a5786-b610-4781-9eb4-583dc599fe7e" (UID: "086a5786-b610-4781-9eb4-583dc599fe7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:34:10.767214 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767170 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-istio-token\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.767214 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767206 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54nvk\" (UniqueName: \"kubernetes.io/projected/086a5786-b610-4781-9eb4-583dc599fe7e-kube-api-access-54nvk\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.767214 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767218 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-envoy\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.767461 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767228 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-credential-socket\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.767461 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767237 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/086a5786-b610-4781-9eb4-583dc599fe7e-istio-data\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.767461 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767245 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/086a5786-b610-4781-9eb4-583dc599fe7e-istiod-ca-cert\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:10.767461 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:10.767254 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/086a5786-b610-4781-9eb4-583dc599fe7e-istio-podinfo\") on node \"ip-10-0-137-56.ec2.internal\" DevicePath \"\"" Jul 18 02:34:11.415160 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.415119 2580 generic.go:356] "Generic (PLEG): container finished" podID="086a5786-b610-4781-9eb4-583dc599fe7e" containerID="a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139" exitCode=0 Jul 18 02:34:11.415335 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.415165 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" Jul 18 02:34:11.415335 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.415206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" event={"ID":"086a5786-b610-4781-9eb4-583dc599fe7e","Type":"ContainerDied","Data":"a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139"} Jul 18 02:34:11.415335 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.415262 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk" event={"ID":"086a5786-b610-4781-9eb4-583dc599fe7e","Type":"ContainerDied","Data":"d223627fa080eb7a7823099cd6ae9a847b024f62176cbb94b50680d72a2126ba"} Jul 18 02:34:11.415335 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.415287 2580 scope.go:117] "RemoveContainer" containerID="a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139" Jul 18 02:34:11.424416 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.424393 2580 scope.go:117] "RemoveContainer" containerID="a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909" Jul 18 02:34:11.431452 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.431423 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk"] Jul 18 02:34:11.434617 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.434427 2580 scope.go:117] "RemoveContainer" containerID="a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139" Jul 18 02:34:11.434814 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:34:11.434797 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139\": container with ID starting with a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139 not found: ID does not exist" containerID="a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139" Jul 18 02:34:11.434879 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.434824 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139"} err="failed to get container status \"a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139\": rpc error: code = NotFound desc = could not find container \"a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139\": container with ID starting with a3cdc261d874d7a44e04fb348d45ea984520427b7ff05da028fc5507a58d8139 not found: ID does not exist" Jul 18 02:34:11.434879 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.434841 2580 scope.go:117] "RemoveContainer" containerID="a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909" Jul 18 02:34:11.435129 ip-10-0-137-56 kubenswrapper[2580]: E0718 02:34:11.435110 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909\": container with ID starting with a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909 not found: ID does not exist" containerID="a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909" Jul 18 02:34:11.435181 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.435134 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909"} err="failed to get container status \"a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909\": rpc error: code = NotFound desc = could not find container \"a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909\": container with ID starting with a1e9b5ee76bd0b1b9da617eb4a85aba48331f9c1140cd0073ee2a8d4f070a909 not found: ID does not exist" Jul 18 02:34:11.435794 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.435769 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-f0c43e-openshift-default-5f95dd75cf-n5msk"] Jul 18 02:34:11.812888 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.812858 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-dxdl4_e662e545-d50a-4f26-9326-04b3bcb0d6ae/copy-manifests/0.log" Jul 18 02:34:11.819629 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:11.819601 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-dxdl4_e662e545-d50a-4f26-9326-04b3bcb0d6ae/manager/0.log" Jul 18 02:34:12.897741 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:12.897706 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" path="/var/lib/kubelet/pods/086a5786-b610-4781-9eb4-583dc599fe7e/volumes" Jul 18 02:34:13.438734 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.438704 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4_b6190791-f1dc-474c-818e-977b822f4338/util/0.log" Jul 18 02:34:13.445410 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.445382 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4_b6190791-f1dc-474c-818e-977b822f4338/pull/0.log" Jul 18 02:34:13.451509 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.451479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4_b6190791-f1dc-474c-818e-977b822f4338/extract/0.log" Jul 18 02:34:13.560464 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.560430 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w_49dd3fd2-70c4-4dda-8843-b4fea0352786/pull/0.log" Jul 18 02:34:13.566609 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.566574 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w_49dd3fd2-70c4-4dda-8843-b4fea0352786/extract/0.log" Jul 18 02:34:13.572697 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.572673 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w_49dd3fd2-70c4-4dda-8843-b4fea0352786/util/0.log" Jul 18 02:34:13.682638 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.682610 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4_4db124ea-af61-4c50-a582-8a4a33ca15db/util/0.log" Jul 18 02:34:13.689659 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.689567 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4_4db124ea-af61-4c50-a582-8a4a33ca15db/pull/0.log" Jul 18 02:34:13.696265 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.696242 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4_4db124ea-af61-4c50-a582-8a4a33ca15db/extract/0.log" Jul 18 02:34:13.800616 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.800570 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p_08d635bd-bdde-4223-912e-c5f69936aba4/pull/0.log" Jul 18 02:34:13.806630 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.806608 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p_08d635bd-bdde-4223-912e-c5f69936aba4/extract/0.log" Jul 18 02:34:13.812219 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:13.812195 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p_08d635bd-bdde-4223-912e-c5f69936aba4/util/0.log" Jul 18 02:34:14.574486 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:14.574458 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-clrdc_41c8968c-4630-4aa9-8a4f-6ebba96dbc5d/limitador/0.log" Jul 18 02:34:15.025041 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:15.025010 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnzwt_70bce6bf-7ca1-467f-a43e-f9872de70801/istio-proxy/0.log" Jul 18 02:34:15.136899 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:15.136862 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jmmc8_020daee4-bbfd-4358-913d-53a36cd4935c/discovery/0.log" Jul 18 02:34:16.559057 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:16.559032 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-st5lq_e31aeab0-beba-4dcf-a184-822682cda2bd/main/0.log" Jul 18 02:34:24.462340 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:24.462307 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rf5pp_bc49c9e2-a220-4c35-979c-0fd30a51cc5e/global-pull-secret-syncer/0.log" Jul 18 02:34:24.592200 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:24.592157 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nt7fw_236ccd4d-b5ea-4d2d-9acf-6d7daefeb8fe/konnectivity-agent/0.log" Jul 18 02:34:24.633870 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:24.633834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-56.ec2.internal_14307c8ae8c30716921eb918c5798d1d/haproxy/0.log" Jul 18 02:34:29.590783 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.590751 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4_b6190791-f1dc-474c-818e-977b822f4338/extract/0.log" Jul 18 02:34:29.615960 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.615927 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4_b6190791-f1dc-474c-818e-977b822f4338/util/0.log" Jul 18 02:34:29.635531 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.635489 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rs6s4_b6190791-f1dc-474c-818e-977b822f4338/pull/0.log" Jul 18 02:34:29.660472 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.660439 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w_49dd3fd2-70c4-4dda-8843-b4fea0352786/extract/0.log" Jul 18 02:34:29.679889 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.679862 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w_49dd3fd2-70c4-4dda-8843-b4fea0352786/util/0.log" Jul 18 02:34:29.701413 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.701383 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mtm4w_49dd3fd2-70c4-4dda-8843-b4fea0352786/pull/0.log" Jul 18 02:34:29.727767 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.727741 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4_4db124ea-af61-4c50-a582-8a4a33ca15db/extract/0.log" Jul 18 02:34:29.751867 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.751839 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4_4db124ea-af61-4c50-a582-8a4a33ca15db/util/0.log" Jul 18 02:34:29.772418 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.772388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73z8db4_4db124ea-af61-4c50-a582-8a4a33ca15db/pull/0.log" Jul 18 02:34:29.798501 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.798474 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p_08d635bd-bdde-4223-912e-c5f69936aba4/extract/0.log" Jul 18 02:34:29.822520 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.822493 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p_08d635bd-bdde-4223-912e-c5f69936aba4/util/0.log" Jul 18 02:34:29.844552 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:29.844475 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15z46p_08d635bd-bdde-4223-912e-c5f69936aba4/pull/0.log" Jul 18 02:34:30.496462 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:30.496435 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-clrdc_41c8968c-4630-4aa9-8a4f-6ebba96dbc5d/limitador/0.log" Jul 18 02:34:32.727879 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:32.727848 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-4qrxk_baaefe17-d29b-4b4c-86e0-ca6692d057b9/monitoring-plugin/0.log" Jul 18 02:34:32.930443 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:32.930401 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rgqvw_339b185b-a593-46e2-9d17-3d8b01689e39/node-exporter/0.log" Jul 18 02:34:32.952419 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:32.952393 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rgqvw_339b185b-a593-46e2-9d17-3d8b01689e39/kube-rbac-proxy/0.log" Jul 18 02:34:32.972400 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:32.972376 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rgqvw_339b185b-a593-46e2-9d17-3d8b01689e39/init-textfile/0.log" Jul 18 02:34:33.000017 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:32.999947 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-lvwng_2602a2ec-ebd8-4034-af72-d6a63cb985d9/kube-rbac-proxy-main/0.log" Jul 18 02:34:33.021350 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:33.021328 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-lvwng_2602a2ec-ebd8-4034-af72-d6a63cb985d9/kube-rbac-proxy-self/0.log" Jul 18 02:34:33.043442 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:33.043414 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-lvwng_2602a2ec-ebd8-4034-af72-d6a63cb985d9/openshift-state-metrics/0.log" Jul 18 02:34:34.342882 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.342842 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr"] Jul 18 02:34:34.343270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343188 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" Jul 18 02:34:34.343270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343199 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" Jul 18 02:34:34.343270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343212 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" Jul 18 02:34:34.343270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343217 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" Jul 18 02:34:34.343270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343227 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15014020-95cb-4ce2-b69a-ef69df250096" containerName="test-curl-7434-140083572479872" Jul 18 02:34:34.343270 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343233 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="15014020-95cb-4ce2-b69a-ef69df250096" containerName="test-curl-7434-140083572479872" Jul 18 02:34:34.343467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343286 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="086a5786-b610-4781-9eb4-583dc599fe7e" containerName="istio-proxy" Jul 18 02:34:34.343467 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.343297 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="15014020-95cb-4ce2-b69a-ef69df250096" containerName="test-curl-7434-140083572479872" Jul 18 02:34:34.346485 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.346465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.348859 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.348827 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-28rtf\"/\"openshift-service-ca.crt\"" Jul 18 02:34:34.349030 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.348868 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-28rtf\"/\"kube-root-ca.crt\"" Jul 18 02:34:34.349441 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.349422 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-28rtf\"/\"default-dockercfg-5hl2v\"" Jul 18 02:34:34.354081 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.354054 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr"] Jul 18 02:34:34.465173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.465125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-podres\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.465396 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.465193 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-sys\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.465396 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.465257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-proc\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.465396 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.465312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-lib-modules\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.465396 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.465340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsgcq\" (UniqueName: \"kubernetes.io/projected/150be9e8-d3db-4422-a88c-6162644820fc-kube-api-access-hsgcq\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566159 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-podres\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-sys\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-proc\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-lib-modules\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsgcq\" (UniqueName: \"kubernetes.io/projected/150be9e8-d3db-4422-a88c-6162644820fc-kube-api-access-hsgcq\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566361 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-podres\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566567 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566358 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-proc\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566567 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566384 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-lib-modules\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.566567 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.566367 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/150be9e8-d3db-4422-a88c-6162644820fc-sys\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.573845 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.573816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsgcq\" (UniqueName: \"kubernetes.io/projected/150be9e8-d3db-4422-a88c-6162644820fc-kube-api-access-hsgcq\") pod \"perf-node-gather-daemonset-fmjsr\" (UID: \"150be9e8-d3db-4422-a88c-6162644820fc\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.616290 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.616215 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-gzhx4_05432b85-ea07-4907-8d79-63c6d1393c1a/networking-console-plugin/0.log" Jul 18 02:34:34.657651 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.657610 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:34.801916 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:34.801887 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr"] Jul 18 02:34:34.804289 ip-10-0-137-56 kubenswrapper[2580]: W0718 02:34:34.804251 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod150be9e8_d3db_4422_a88c_6162644820fc.slice/crio-32259b5a6d43472704588c4785f9524791e3db4a10a49a22a7346055373a2f21 WatchSource:0}: Error finding container 32259b5a6d43472704588c4785f9524791e3db4a10a49a22a7346055373a2f21: Status 404 returned error can't find the container with id 32259b5a6d43472704588c4785f9524791e3db4a10a49a22a7346055373a2f21 Jul 18 02:34:35.544724 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:35.544677 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" event={"ID":"150be9e8-d3db-4422-a88c-6162644820fc","Type":"ContainerStarted","Data":"ed07c24dd21c41dee40d3c182721af9af6a7703e84a096877d7b14204037c50b"} Jul 18 02:34:35.545173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:35.544748 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" event={"ID":"150be9e8-d3db-4422-a88c-6162644820fc","Type":"ContainerStarted","Data":"32259b5a6d43472704588c4785f9524791e3db4a10a49a22a7346055373a2f21"} Jul 18 02:34:35.545173 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:35.544791 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:35.777861 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:35.777817 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7f8db66bcb-vvht9_fbf89c88-bc0c-457b-aa05-366ed0e8c39d/console/0.log" Jul 18 02:34:35.808391 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:35.808291 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-7crkn_725cba84-708c-47ff-ab2b-8d54c646ec0e/download-server/0.log" Jul 18 02:34:37.329677 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:37.329559 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b6h62_7d8e622b-fcde-4197-b1da-670a26b2f3e4/dns/0.log" Jul 18 02:34:37.350347 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:37.350321 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b6h62_7d8e622b-fcde-4197-b1da-670a26b2f3e4/kube-rbac-proxy/0.log" Jul 18 02:34:37.457644 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:37.457619 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xsvd_50f3cf71-8c8d-40af-8004-0b7886594448/dns-node-resolver/0.log" Jul 18 02:34:38.032119 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:38.032064 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c5d7dd6cb-qtz9j_5c2d9f85-0c6e-4a1a-8359-20464a09bddb/registry/0.log" Jul 18 02:34:38.074134 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:38.074092 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87z6p_891e9aab-d18d-4130-94b5-e1ec8af5717b/node-ca/0.log" Jul 18 02:34:38.979291 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:38.979258 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnzwt_70bce6bf-7ca1-467f-a43e-f9872de70801/istio-proxy/0.log" Jul 18 02:34:39.112661 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:39.112627 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jmmc8_020daee4-bbfd-4358-913d-53a36cd4935c/discovery/0.log" Jul 18 02:34:39.945191 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:39.945160 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l8w46_aa8c1918-c53a-46bb-a01d-d508b6e8f7c2/serve-healthcheck-canary/0.log" Jul 18 02:34:40.470892 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:40.470858 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cwtbf_afbf040c-2b74-48d7-a323-55739ec6a3f0/kube-rbac-proxy/0.log" Jul 18 02:34:40.493747 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:40.493716 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cwtbf_afbf040c-2b74-48d7-a323-55739ec6a3f0/exporter/0.log" Jul 18 02:34:40.514998 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:40.514967 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cwtbf_afbf040c-2b74-48d7-a323-55739ec6a3f0/extractor/0.log" Jul 18 02:34:41.559557 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:41.559524 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" Jul 18 02:34:41.576019 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:41.575959 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-fmjsr" podStartSLOduration=7.575940299 podStartE2EDuration="7.575940299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:35.559447258 +0000 UTC m=+2781.243320001" watchObservedRunningTime="2026-07-18 02:34:41.575940299 +0000 UTC m=+2787.259813038" Jul 18 02:34:43.225947 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:43.225913 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-dxdl4_e662e545-d50a-4f26-9326-04b3bcb0d6ae/manager/0.log" Jul 18 02:34:43.247707 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:43.247676 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-dxdl4_e662e545-d50a-4f26-9326-04b3bcb0d6ae/copy-manifests/0.log" Jul 18 02:34:44.744792 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:44.744746 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6bc78dbd6c-mx97h_4360e211-4224-4eed-a262-50aba312e148/manager/0.log" Jul 18 02:34:44.768885 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:44.768850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-xrmvl_dba61589-c1ca-4ddb-aa83-1520f5a50519/openshift-lws-operator/0.log" Jul 18 02:34:45.454269 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:45.454235 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnzwt_70bce6bf-7ca1-467f-a43e-f9872de70801/istio-proxy/0.log" Jul 18 02:34:45.574175 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:45.574143 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jmmc8_020daee4-bbfd-4358-913d-53a36cd4935c/discovery/0.log" Jul 18 02:34:46.864234 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:46.864207 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bnzwt_70bce6bf-7ca1-467f-a43e-f9872de70801/istio-proxy/0.log" Jul 18 02:34:46.987596 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:46.987551 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-jmmc8_020daee4-bbfd-4358-913d-53a36cd4935c/discovery/0.log" Jul 18 02:34:49.720239 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:49.720209 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-zn2wb_2c18e87e-8502-4328-9d2b-8c9fa9feb8b6/migrator/0.log" Jul 18 02:34:49.754359 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:49.754322 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-zn2wb_2c18e87e-8502-4328-9d2b-8c9fa9feb8b6/graceful-termination/0.log" Jul 18 02:34:51.263920 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.263894 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/kube-multus-additional-cni-plugins/0.log" Jul 18 02:34:51.285043 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.285009 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/egress-router-binary-copy/0.log" Jul 18 02:34:51.304162 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.304137 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/cni-plugins/0.log" Jul 18 02:34:51.322722 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.322695 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/bond-cni-plugin/0.log" Jul 18 02:34:51.342425 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.342402 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/routeoverride-cni/0.log" Jul 18 02:34:51.361656 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.361631 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/whereabouts-cni-bincopy/0.log" Jul 18 02:34:51.381126 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.381101 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dqb6d_a79aa2c5-6ecd-4434-a381-adc4351be14a/whereabouts-cni/0.log" Jul 18 02:34:51.604345 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.604239 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qb2lp_e9af9005-775c-4224-9567-909f4795d169/kube-multus/0.log" Jul 18 02:34:51.623501 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.623468 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4xj87_19f50b49-b7d3-4df2-9619-4464c6bf21ba/network-metrics-daemon/0.log" Jul 18 02:34:51.644188 ip-10-0-137-56 kubenswrapper[2580]: I0718 02:34:51.644153 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4xj87_19f50b49-b7d3-4df2-9619-4464c6bf21ba/kube-rbac-proxy/0.log"